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REAL PARTY IN INTEREST: 

The real party in interest is tlie assignee of the application, Siemens 
Aktiengesellschaft, a German corporation. 
RELATED APPEALS AND INTERFERENCES: 

There are no related appeals and no related interferences. 
STATUS OF CLAIMS: 

Claims 2 and 10-14 are the subject of the present appeal, and constitute all 
pending claims of the application. Each of claims 2 and 10-14 stands as being finally 
rejected in the Office Action dated March 23, 2010. 

Claims 1 and 3-9, which were previously pending in the application, are 
cancelled. 

STATUS OF AMENDMENTS: 

No Amendment was filed following the Final Rejection. 
SUMMARY OF CLAIMED SUBJECT MATTER: 

Independent claim 10 is the only independent claim of the application, and is 
set forth below with exemplary citations to the specification and drawings for all 
relevant claim limitations therein. 

10. An operating device for a medical diagnostic imaging unit, said 
operating device comprising: 

a display screen (touchscreen having a display area 20 and operating device 

1 1 in Figs 2 and 3, p. 6, 1. 11-12 and I. 21-22); 
a control unit (operating device 11) configured to operate said display screen 
to display on said display screen one and only one at a time of a 
display area and an operating area (Figs. 2 and 3), that do not overlap 
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each other (areas 20 and 21 do not overlap), and a mode selection 
field (keys 23 in Figs 2 and 3; p. 7, 1.3 - p.8, 1.4) and also being 
configured to enter, in a current-value entering session, at least one 
examination value for implementing an examination by said medical 
diagnostic imaging unit; 

said control unit being configured to operate said display screen, in said 
current-value entering session, in a programmed mode in which, in 
said operating area of the display screen, only a selection key field is 
displayed (selection keys 14, 15, 16 in Figs 2 and 3; p. 6, I. 16-20), said 
selection key field being activatable to select at least one preset value 
that is preset prior to said current value entering session (p. 7, I. 16- 
17), said at least one preset value being selected from the group 
consisting of preset operating values of said medical diagnostic 
imaging unit and preset parameter values of said medical diagnostic 
imaging unit (p. 6, 1 . 21-23); 

said control unit being also configured to operate said display screen , in said 
current-value entering session, in a manual mode (p. 7, 1.30 - p.8, 1. 4) 
in which, in said operating area of said display screen, only a setting 
key field is displayed (setting keys 14, 15, 16 in Figs. 2 and 3; p.8, I. 
24-26), said setting key field being activatable to selectively set at least 
one settable value selected from the group consisting of settable 
operating values of said medical diagnostic imaging unit and settable 
parameters of said medical diagnostic imaging unit (p.8, 1.12-23); 



-4- 



said control unit being configured to display, in said display area in said 
current-value entering session, display elements (display elements 19 
in Figs 2 and 3) respectively representing said at least one preset value 
and said at least one settable value (p.6, I. 11-18); 

said control unit being configured to display at said display screen, in said 
current-value entering session, said mode selection field (switchover 
keys 23 in Figs 2 and 3), said mode selection field being activatable to 
select, as a selected mode, only one of either, said manual mode or 
said programmed mode (p. 7, I. 30 - p. 8, 1.4); 

said control unit being configured, in said current-value entering session, to 
initially maintain all of said display area unchanged and visually 
unobstructed with said at least one preset value or said at least one 
settable value displayed only once and only in said display area, when 
switching between said manual mode and said programmed mode by 
activation of said mode selection field, until said selection key field or 
said setting key field in the selected mode is activated after said 
switching (p. 9, I. 3-11); and 

said control unit being configured to display, in said current-value entering 
session, at said display screen, a trigger key (recording key 24 in Figs. 
2 and 3) that, when activated, emits a current content of said display 
area, as said at least one examination value, as an output available to 
said medical diagnostic imaging unit (p.7, I. 12-15 and p.9, I. 12-19). 

Figs 1-3 of the application as originally filed are submitted herewith as Exhibit 



GROUND OF REJECTION TO BE REVIEWED ON APPEAL: 

The sole issue to be reviewed on appeal is whether the subject matter of 
claims 2 and 10-14 would have been obvious to a person of ordinary skill in the field 
of designing operating systems for a medical examination apparatus, under the 
provisions of 35 U.S.C, §1 03(a) based on the teachings of United States Patent No. 
6,795,528 (Nokita, Exhibit "B") in view of United States Patent No. 5,675,755 
(Trueblood, Exhibit "C"), United States Patent No. 6,847,336 (Lemelson et al., 
Exhibit "D"), United States Patent No. 5,999,177 (Martinez, Exhibit "E"), United 
States Patent No. 6,983,331 (Mitchell et al., Exhibit "F"), and United States Patent 
No. 6,674,449 (Banks et al., Exhibit "G"). 
ARGUMENT: 

Rejection of Claims 2 and 10-14 Under 35 U.S.C. §103(a) Based on Nokita, 
Trueblood, Lemelson et al, Martinez, Mitchell et al. and Banks et al, 

In the Final Rejection dated March 23, 2010, in substantiating the above 
rejection, the Examiner stated that the Nokita reference discloses all of the 
linnitations of independent claim 10, but acknowledged that the setting key field 
shown in Figure 8C of Nokita defers from the claimed invention insofar as the pop-up 
window does not completely overlap the operating area (Figure 8A; 840), and it 
overlaps some of the display area (Figure 8a, 830, 835, 825). The Examiner also 
acknowledged that the pop-up window disclosed in the Nokita reference displays the 
same parameter values (voltage, current, time and focal length) as were displayed in 
the display area, prior to the window popping-up and covering them. 

The Examiner stated it would have been an obvious design choice to a 
person of ordinary skill in the art to move or re-size the pop-up window in the Nokita 
reference so that it would overlap the entire operating area, but not overlap any of 
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the display area, so as to provide the user with a clear view of all of the pertinent 
information on the entire left half of the screen, while simultaneously preventing the 
user from inadvertently pressing any of the operating area buttons. The Examiner 
stated such a repositioning of the pop-up window would reveal all of the parameter 
values in the display area, thereby eliminating any need for the pop-up window to 
display those values a second time. The Examiner also stated it would have been 
an obvious design choice to a person of ordinary skill in the art to remove duplicate 
data from being displayed, so as to save screen real estate, lower graphical 
processing demands, and present a less cluttered display image for the user. 

The Examiner stated these modifications would have been obvious to a 
person of ordinary skill in the art because such a person would have "good reason" 
to pursue the known options within his or her technical grasp. 

The Examiner relied on the Trueblood reference as disclosing a method for 
automatically preventing a pair of windows from overlapping each other. The 
Examiner relied on the Lemelson et al. reference as disclosing a graphical user 
interface for displaying x-ray images next to operating device parameter values. 

The Examiner relied on the Martinez reference as disclosing a method for 
hiding display information that "serves no useful purpose" for the user. 

The Examiner relied on the Mitchell et al. reference as disclosing a method for 
prioritizing sets or areas of display content and suppressing the display of some of 
that content when the available display space is determined to be too small. 

The Examiner relied on the Banks et al. reference as disclosing a touch panel 
trigger key that causes a medical image to take a picture of a patient. 
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Appellants respectfully submit that the Examiner has used Appellants' 
disclosure as a roadmap through their prior art in order to suggest re-designing the 
display disclosed in the Nokita reference, and it is only by being "told" by Appellants' 
disclosure, as reflected in the claim language, that the explicit teaching in the Nokita 
reference to display respective values twice, and at two different locations, can be 
ignored and/or modified. It is important in the subject matter of claim 10 that when 
switching between the manual mode and the program mode, the value that is 
displayed at the display area is displayed only once and at only one location, i.e., in 
the display area. Even with the resizing of the display area of Nokita proposed by 
the Examiner in order to modify the operation of the Nokita reference, the respective 
values still would be displayed twice, and at two different locations. 

This is even more relevant due to the further language in claim 10 that 
requires that, in the programmed mode, only a selection key field is displayed and, in 
the manual mode, only a setting key field is displayed. It is therefore not permissible 
for the Examiner to ignore the display area in Nokita, because this contradicts the 
requirement of claim 10 that only the aforementioned fields be displayed. 

The Examiner is relying on the Lemelson et al. reference as providing a 
teaching of initially maintaining all of the display are unchanged and visually 
unobstructed, when switching between a manual mode and a programmed mode. 

Appellants respectfully submit that even if such a teaching could be found in 
the Lemelson et al. reference, there is no basis to modify the Nokita reference by 
making use of such an alleged teaching. This is for the following reasons. 
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From Figures 8A and 8B of Nokita, it is clear that a number of parameters 835 
(72 kV, 170 mA, 50 msec, 120 cm) are shown. In Figure 8C of Nokita, however, only 
72 kV remains (at least partially) visible. 

Therefore, Appellants submit that the Nokita reference does not disclose 
"maintaining all of said display area unchanged and visually unobstructed" as 
required in claim 10. 

The Examiner has maintained the position that a resizing of the window in the 
foreground of Figure 8C would render all of the display area visible. Such a resizing 
was state of the art at the time of the present application, as disclosed in Lemelson 
et al. The Examiner uses this information as a basis for combining Nokita and 
Lemelson et al. 

Appellants submit that the Nokita reference teaches away from employing 
such a resizing of the foreground window of Figure 8C. This is for the following 
reasons. 

The aforementioned parameters 835 shown in Figures 8A and 8B are shown 
in Figure 8C at a different location, which is approximately in the center of the 
foreground window. If the window were resized according to the Examiner's 
suggestion, the set of parameters would be shown twice. Since it would be 
redundant to show the parameters a second time, this would not be a modification 
that a person of ordinary skill in the art would consider reasonable. Moreover, such 
a redundant display of the parameters would be confusing and detract from the user- 
friendliness of the display. A user does not expect information to be shown twice 
without any logical reason. Moreover, the user would not expect a background 
window to provide relevant information, because the user knows that the size of 
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display windows is changeable, and is therefore unlikely to pay much attention or 
attach importance to information shown in the background window. In addition to the 
redundancy, these factors would dissuade a person of ordinary skill in the art from 
undertaking the type of resizing proposed by the Examiner. 

Additionally, resizing of the foreground window would cause additional 
information from the background window to be displayed that does not have 
significance in the current mode. This is because windows generally assume an 
oblong shape, and a resizing such that the entirety of the parameters 835 are still 
visible in the mode of Figure 8C would unavoidably result in additional information 
being shown from the background window that are not useful or needed in the 
current mode, and this would further detract from the user-friendliness. Those of 
ordinary skill in the field of designing computer displays operate on the premise that 
only relevant information should be shown at any given time. If the resizing were to 
reduce the height of the window, a setting key field (at the right side of Figures 8A 
and SB) may even be visible in the mode of Figure 8C. This is also counter to the 
basis of the subject matter disclosed and claimed in the present application, which is 
for the purpose of providing a visually clear separation between setting keys and 
selection keys. 

For these reasons. Appellants respectfully submit that the resizing proposed 
by the Examiner has occurred to the Examiner only as a way to "force" the 
Nokita/Lemelson et al. combination to allegedly conform to the subject matter of 
claim 10. It is not a practical or reasonable modification that would be considered by 
a person of ordinary skill in the art. 



The Federal Circuit stated in In re Lee 227 F.3d 1338, 61 U.S.P.Q. 2d 1430 



(Fed. Cir. 2002) : 

"The factual inquiry whether to combine references must be thorough 
and searching. ...It must be based on objective evidence of record. 
This precedent has been reinforced in myriad decisions, and cannot be 
dispensed with." 

Similarly, quoting C.R. Bard. Inc. v. M3 Systems, Inc., 157 F.3d 1340, 1352, 

48 U.S.P.Q. 2d 1225, 1232 (Fed. Cir. 1998) , the Federal Circuit in Brown & 

Williamson Tobacco Court v. Philip Morris, Inc., 229 F.3d 1120, 1124-1125, 56 

U.S.P.Q. 2d 1456, 1459 (Fed. Cir. 2000) stated: 

[A] showing of a suggestion, teaching or motivation to combine the 
prior art references is an 'essential component of an obviousness 
holding'. 

In In re Dembiczak, 175 F.3d 994,999, 50 U.S.P.Q. 2d 1614, 1617 (Fed. Cir. 

1999) the Federal Circuit stated: 

Our case law makes clear that the best defense against the subtle but 
powerful attraction of a hindsight-based obviousness analysis is 
rigorous application of the requirement for a showing of the teaching or 
motivation to combine prior art references. 

Consistently, in In re Rouffet 149 F.3d 1350, 1359, 47 U.S.P.Q. 2d 1453, 

1459 (Fed. Cir. 1998) , the Federal Circuit stated: 

[EJven when the level of skill in the art is high, the Board must identify 
specifically the principle, known to one of ordinary skill in the art, that 
suggests the claimed combination. In other words, the Board must 
explain the reasons one of ordinary skill in the art would have been 
motivated to select the references and to combine them to render the 
claimed invention obvious. 

In Winner International Royalty Corp. v. Wang, 200 F.3d 1340, 1348-1349, 53 

U.S.P.Q. 2d 1580, 1586 (Fed. Cir. 2000) , the Federal Circuit stated: 

Although a reference need not expressly teach that the disclosure 
contained therein should be combined with another, ... the showing of 
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comblnability, in whatever form, must nevertheless be clear and 
particular. 

Lastly, in Crown Operations International, Ltd, v, Solutia. Inc.. 289 F.3d 1367, 

1376. 62 U.S.P.Q. 2d 1917 (Fed. Cir. 2002) . the Federal Circuit stated: 

There must be a teaching or suggestion within the prior art, within the 
nature of the problem to be solved, or within the general knowledge of 
a person of ordinary skill in the field of the invention, to look to 
particular sources, to select particular elements, and to combine them 
as combined by the inventor. 

Appellants submit that the decision of the United States Supreme Court in 
KSR International Co, v. Teleflex Inc.. 550 U.S. 398. 127 S.Ct. 1727. 82 USPQ 2d 
1385 (2007) , and the United States Patent and Trademark Office guidelines for 
applying that decision, support the position of the Appellants. That decision, 
although stating that it is not always required to point to a specific teaching in a prior 
art reference in order to substantiate a rejection under 35 U.S.C. §1 03(a), by no 
means approved ignoring the above long-standing precedent, and certainly did not 
represent a blanket overruling of that precedent. In the KSR decision, the Supreme 
Court stated, under certain circumstances, it may not be necessary to point to a 
specific passage in a prior art reference as evidence of motivation, guidance or 
inducement in order to modify that reference in a manner that obviates the patent 
claim in question. The Supreme Court stated that if a person of ordinary skill in the 
art can implement a predictable variation and would see the benefit of doing so. 
Section 103(a) likely bars patentability. 

Nevertheless, the Supreme Court also stated that the requirement to find a 
teaching, suggestion or motivation in the prior art "captures a helpful insight." The 
Supreme Court stated that although common sense directs caution as to a patent 



application claiming as innovation the combination of two known devices according 
to their established functions, it can be important to identify a reason that would have 
prompted a person of ordinary skill in the art to combine the elements as the new 
invention does. The Supreme Court, however, stated that not every application 
requires such detailed reasoning. The Supreme Court stated that helpful insights 
need not become rigid and mandatory formulas. The Supreme Court only stated 
that if the requirement to find a teaching, suggestion or motivation is required in such 
a rigid, formulaic manner, it is then inconsistent with the precedence of the Supreme 
Court. In fact, the Supreme Court stated that since the "teaching, suggestion or 
motivation" test was devised, the Federal Circuit doubtless has applied it in accord 
with these principles in many cases. The Supreme Court stated there is no 
necessary inconsistency between this test and an analysis conducted under the 
standards of Graham v. Deere. The Supreme Court stated the only error is 
transforming this general principle into a "rigid rule limiting the obviousness inquiry." 

Therefore, Appellants submit this decision of the Supreme Court does not in 
any manner approve, much less require, the absence of a rigorous evidentiary 
investigation on the part of the Examiner in order to substantiate most rejections 
under 35 U.S.C. §1 03(a). Only under the somewhat unusual, and very limited, 
circumstances outlined by the Supreme Court in the KSR decision might the 
Supreme Court excuse the absence of such a rigorous evidentiary investigation in 
reaching a conclusion of obviousness under 35 U.S.C. §1 03(a). 

This view of the KSR decision has been substantiated by the United States 
Court of Appeals for the Federal Circuit in Takeda Chemical Industries Limited v. 
AlDhapharm Ptv.Ltd.A92 F.3d 1350, 83 U.S.P.Q.2d, 169 (Fed. Cir 2007) . which was 

-13- 



one of the earliest decisions of the Federal Circuit after the KSR decision was 

decided by the Supreme Court. The Takeda decision concerned a chemical patent 

that was the subject of an infringement lawsuit, and which was attacked by the 

infringer on the basis of the claimed subject matter being "obvious to try." After 

acknowledging that the KSR decision held that the teaching-suggestion-motivation 

test should not be applied rigidly, the Federal Circuit stated that the KSR decision 

actually recognized the value of that test in determining whether the prior art 

provided a reason for one of skill in the art to make the claimed combination. The 

Federal Circuit stated this is consistent with the Federal Circuit precedent in In re 

Dillon, 919 F.2d 688 (Fed. Cir. 1990) and in In re DeueL 51 F.3d 1552 (Fed. Cir. 

1995) . The Federal Circuit stated that in cases involving new chemical compounds, 

it remains necessary to identify some reason that would have led a chemist to modify 

a known compound in a particular manner to establish prima facie obviousness of 

the new claimed compound. In the Takeda decision, the Federal Circuit stated: 

The KSR Court recognized that "[w]hen there is a design need or 
market pressure to solve a problem and there are a finite number of 
identified, predictable solutions, a person of ordinary skill has good 
reason to pursue the known options within his or her technical grasp," 
KSR. 127 S.Ct. at 1732 . In such circumstances, "the fact that a 
combination was obvious to try might show that it would obvious under 
§103." id that is not the case here. Rather than identify predictable 
solutions for antidiabetic treatment, the prior art disclosed a broad 
selection of compounds, any one of which could have been selected as 
a lead compound for further investigation. 

For the above reasons, Appellants submit that the modification of Nokita in 
view of Lemelson et al. proposed by the Examiner would not have been obvious to a 
person of ordinary skill in the field of designing operating systems for medical 
examination installations, under the provisions of 35 U.S.C. §1 03(a). It is not 
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necessary to individually address the other secondary references relied upon by the 
Examiner, because it is the position of the Appellants that the Nokita/Lemelson et al. 
combination does not disclose or suggest the limitations of claim 10 against which 
that combination was applied, and therefore even if the Nokita/Lemelson et al. 
combination were further modified in accordance with any of the additional 
references relied upon by the Examiner, the subject matter of claim 10 still would not 
result. 

The same arguments apply to dependent claims 2 and 11-14 that depend 
from claim 10. None of those claims would have been obvious to a person of 
ordinary skill in the field of designing operating systems for medical examination 
installations for the same reasons discussed above in connection with claim 10. 
CONCLUSION: 

For the above reasons, Appellants respectfully submit the Examiner is in error 
in law and in fact in rejecting claims 2 and 10-14 as being obvious under 35 U.S.C. 
§ 103(a) based on the teachings of the above-cited references. Reversal of that 
rejection is proper, and the same is respectfully requested. 

This Appeal Brief is accompanied by electronic payment for the requisite fee 
in the amount of $540.00. 
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Patent Department 
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CLAIMS APPENDIX 

2. The operating device as claimed in claim 10, wherein the operating 
device is designed as a touch-sensitive display screen. 

10. An operating device for a medical diagnostic imaging unit, said 
operating device comprising: 

a display screen; 

a control unit configured to operate said display screen to display on said 
display screen one and only one at a time of a display area and an 
operating area, that do not overlap each other, and a mode selection 
field and also being configured to enter, in a current-value entering 
session, at least one examination value for implementing an 
examination by said medical diagnostic imaging unit; 

said control unit being configured to operate said display screen, in said 
current-value entering session, in a programmed mode in which, in 
said operating area of the display screen, only a selection key field is 
displayed, said selection key field being activatable to select at least 
one preset value that is preset prior to said current value entering 
session, said at least one preset value being selected from the group 
consisting of preset operating values of said medical diagnostic 
imaging unit and preset parameter values of said medical diagnostic 
imaging unit; 

said control unit being also configured to operate said display screen , in said 
current-value entering session, in a manual mode in which, in said 
operating area of said display screen, only a setting key field is 
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displayed, said setting key field being activatable to selectively set at 
least one settable value selected from the group consisting of settable 
operating values of said medical diagnostic imaging unit and settable 
parameters of said medical diagnostic imaging unit; 
said control unit being configured to display, in said display area in said 
current-value entering session, display elements respectively 
representing said at least one preset value and said at least one 
settable value; 

said control unit being configured to display at said display screen, in said 
current-value entering session, said mode selection field, said mode 
selection field being activatable to select, as a selected mode, only one 
of either, said manual mode or said programmed mode; 

said control unit being configured, in said current-value entering session, to 
initially maintain all of said display area unchanged and visually 
unobstructed with said at least one preset value or said at least one 
settable value displayed only once and only in said display area, when 
switching between said manual mode and said programmed mode by 
activation of said mode selection field, until said selection key field or 
said setting key field in the selected mode is activated after said 
switching; and 

said control unit being configured to display, in said current-value entering 
session, at said display screen, a trigger key that, when activated, 
emits a current content of said display area, as said at least one 



examination value, as an output available to said medical diagnostic 
imaging unit. 

1 1 . The operating device as claimed in claim 10 wherein said control unit is 
configured to display said display elements as text elements. 

12. The operating device as claimed in claim 10 wherein said control unit is 
configured to display said display elements as graphics elements. 

13. The operating device as claimed in claim 10 wherein said control unit is 
configured to display said trigger key at said display screen in each of said manual 
mode and said programmed mode. 

14. The operating device as claimed in claim 10 wherein said medical 
diagnostic imaging unit is an x-ray examination unit, and wherein said control unit is 
configured to display, in said selection key field in said programmed mode, a plurality 
of selection keys each associated with one anatomical x-ray examination in a 
plurality of anatomical x-ray examinations, each selection key allowing a user to 
select said at least one preset value for the anatomical x-ray examination associated 
with that selection key, and to display, in said setting key field in said manual mode, 
a plurality of different setting keys that respectively allow manual setting of said at 
least one settable value for a component of said x-ray examination unit. 
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EVIDENCE APPENDIX 

Exhibit A: Figs 1-3 as originally filed on December 1 , 2003 

Exhibit B: United States Patent No. 6,795,528 - cited in the March 23, 

2010 Final Rejection 
Exhibit C: United States Patent No. 5,675,755 - cited in the March 23, 

2010 Final Rejection 
Exhibit D: United States Patent No. 6,847,336 - cited in the March 23, 

2010 Final Rejection 
Exhibit E: United States Patent No. 5,999,177 - cited in the March 23, 

2010 Final Rejection 
Exhibit F: United States Patent No. 6,983,331 - cited in the March 23, 

2010 Final Rejection 
Exhibit G: United States Patent No. 6,674,449 - cited in the March 23, 

2010 Final Rejection 
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RELATED PROCEEDINGS APPENDIX 



None. 
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ABSTRACT 



A radiographic apparatus is disclosed which controls a 
movement of a reciprocatingly moving grid so that the grid 
is not or less likely returned in the middle of exposure of an 
object to X rays. The probability that the object is still 
exposed to the X rays when the grid is moved in the vicinity 
of a turning point is thus substantially lowered. Therefore 
the probability that a resulting radiograph has no or less 
moire pattern due to the grid is substantially heightened. 

5 Claims, 9 Drawing Sheets 
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RADIOGRAPHIC APPARATUS, The above-mentioned digital radiographic apparatuses 

RADIOGRAPHIC METHOD, AND present an X-ray absorption ratio higher than that of the 

COMPUTER-READABLE STORAGE conventional film/screen system, and is free from a mottle 

MEDIUM structure that degrades granularity of an image in the con- 

5 ventional film/screen system. An X-ray image obtained from 
the above-mentioned digital radiographic apparatus is a 

BACKGROUND OF THE INVENTION digital image. Therefore the X-ray image is not necessarily 

1 Field of the Invention output with a predetermined tone scale characteristics the 

Tlie present invention relates to a radiographic apparatus, f?""^ f ^^^^ ^ .f ^1^/^^^^^° system. Observing an 

a radiographic method, and a computer-readable recording X-ray image on a monUor, 

medium, which can use a scattered radiation removal grid '"'^^ charactenstics The user is also free to perform image 
for removing radiation scattered by and within an object V^^^f^sing such as frequency analysis, frequency enhance- 
when the object is radiographed. 1^"°^ ^5 ^^VVr^^ion processmg or the like on the resulting 
« ^ . .. J. ^1.1^ X-ray image, thereby improving image quality, 
2. Descnption of the Related Art ^.^ ^ . • *u u 
^ J. , . , ,r . . 1 . .15 The digital radiographic apparatus using the above- 
Radiographing such as X-ray imaging has been used m mentioned scattered X ray removal grid suffers from a moire 
diagnosis m medical practice, and examination and test of ^^^^^^ ^^riped pattern, here) which does not exist in the 
industrial products, for example. In X-ray imaging, an object x-ray image in the conventional film/screen system. The 
is exposed to X rays, and X rays transmitted through the ^^^ire pattern is generated on the basis of a difference 
object are used to photosensitize a film or a screen. An X-ray ^^^^^^^ p^^^j p-^^j^ ^ ^^^^^^ X ray as 
image is thus obtamed. The X rays transmitted through the ^n electrical signal and the pitch of the lead foils forming the 
object contain X rays Hnearly transmitted through the object (the reciprocal number of the grid density). Let Ny 
and X rays that have been scattered withm the object and represent the Nyquist frequency of the sensor and Gy 
then transmitted therethrough (hereinafter referred to as ^present the grid frequency (the grid density), and the 
scattered X rays ). In X ray imaging, the scattered X rays 25 frequency of the moire pattern is expressed as 2 Ny-Gy. 
degrade the contrast of an X ray image, thereby substantially ^ ^ ^^^^ ^ ^ ^ ^.5 
affecting the quality of the X-ray image, jp/^^ ^^^^ p.^^1 p .^^^^ ^^^^ 200 Wand the grid 
A scattered X ray removal gnd (hereinafter sunply frequency Gy is 4.0 Ip/mm (40 lines/cm), the frequency of 
referred to as a "grid") is arranged parallel to the plane of the moire pattern is 1.0 Ip/mm. Since this frequency com- 
imagmg to efficiently remove the scattered X rays thereby 30 ponent is the one appearing in the typical X-ray images, the 
improving the contrast of the X-ray image. The gnd is ^oire pattern is distinctly visible, thereby substantially 
formed of lead foils that are arranged so as to focus toward degrading the quality of the resulting X-ray images. Here, 
a focal point of an X-ray source, or lead foils that are ip/^jm stands for line pairs per mm. The resolving power of 
arranged perpendicular to the imaging plane. The grid the radiographic apparatus is sometimes measured using a 
facilitates the transmission of X rays that have entered after 35 ^hart having lead lines. The resolving power is defined by 
being linearly transmitted through the object from the focal referring to how many pairs of the presence and absence of 
point of the X-ray source while blocking the X rays that are the lead Hne are recognized per unit length, 
incident at a slant angle after being scattered within the ^-^^ ^^^^^^^ of technology, the digital radiographic 
^ ^ ' apparatus provides an X-ray absorption ratio and resolving 
In the conventional X-ray imaging using a film/screen 40 power higher than those of the conventional film/screen 
system with the grid, the frequency of stnpe-like lead foils radiographic apparatus using the mm/screen system, View- 
of gnd (i.e., the reciprocal number of the pitch of the lead ing the screen of a monitor, the user is free to change the tone 
foils, also referred to as a "grid frequency" or a "grid scale of the X-ray image. Therefore the component of the 
density") IS typically set higher than a frequency component moire pattern having some contrast, as opposed in the 
of an ordinary X-ray image (on the assumption that no grid 45 conventional X-ray imaging, becomes a problem, 
is employed) so that a striped pattern, occurring due to the Accordingly, a movement method of the grid different from 
grid in the X-ray image, is inconspicuous. By moving the that in the conventional film/screen system and a radio- 
gnd m parallel to the imaging plane, the contrast of the graphing method accounting for the movement method are 
striped pattern occurrmg due to the grid is reduced to make needed to make the moire pattern removed or inconspicu- 
the striped pattern less visible or invisible in the X-ray 50 ous. 
image. 

Digital radiographic apparatuses have been used. The SUMMARY OF THE INVENTION 

digital radiographic apparatus obtains X-ray image data by Accordingly, it is an object of the present invention to 

converting the X rays transmitted through the object into provide a radiographic apparatus, a radiographic method, 

visible light rays proportional to the intensity of the X rays 55 and a computer- readable storage medium for acquiring a 

through a fluorescent material (scintillator), converting the radiograph, wherein the probability that a moire pattern is 

visible light rays into electrical signals through a photoelec- not generated or, is inconspicuous in the radiograph is 

trie conversion element comprising a plurality of pixels, and heightened. 

(analog-to-digital) converting the electrical signals by an AD In one aspect of the present invention, a radiographic 

converter. Digital radiographic apparatuses of a sort begin to 60 apparatus for obtaining a radiograph of an object includes a 

be used which obtain X-ray image data by sensing the grid movement controller for controlling a movement of a 

distribution of electric charges proportional to the intensity grid which can move reciprocatingly, an input unit for 

of X rays that result from directly absorbing (through inputting a method parameter relating to a radiographic 

photoelectric conversion) X rays transmitted through an method, and an imaging controller for setting a movement 

object, converting the charges into electrical signals, and 65 parameter relating to the movement of the grid to be used by 

(analog-to-digital) converting the electrical signals through said grid movement controller based on the method param- 

an AD converter. eter input by the input circuit. 



of the present invention, 

FIG. 2 is a graph plotting the interrelationship between an 
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In another aspect of the present invention, a radiographic measurement by the grid position measurement unit 175 and 
method for obtaining a radiograph of an object, includes the a command from an imaging controller 180, the grid move- 
steps of controlling a movement of a grid which can move ment controller 170 controls the movement of the grid 130 
reciprocatingly, inputting a method parameter relating to a while assuring alignment with the timing of the X-ray 
radiographic method, and setting a movement parameter 5 imaging. The grid position measurement unit 175 under the 
relating to the movement of the grid to be used in said grid control of the grid movement controller 170 measures the 
movement controlling step based on the method parameter position of the grid 130. 

input in said inputting step. The imaging controller 180 controls X-ray imaging by 

Further objects, features, and advantages of the present commanding that the grid 130 should be moved and by 

invention will be apparent from the following description of 10 driving the sensor 140. An image processor 190 processes an 

the preferred embodiments with reference to the attached X-ray image acquired by the X-ray image acquisition unit 

drawings. 150 so that a user may easily recognize the image. An image 

display unit 200 displays the X-ray image processed by the 

BRIEF DESCRIPTION OF THE DRAWINGS image processor 190. An image storage unit 240 stores the 

r^r^ ^ . ui t J- u ' V J- 1 15 X-ray image processed by the image processor 190. 

FIG. 1 IS a block diagram showmg an X-ray digital t^ . ^^ , 

radiographic apparatus in accordance with one embodiment . P^^^^^°f ^"^^^ ^""^L^^^ ^""^^ 

unagmg. An imagmg operation umt 220, including an opera- 
tion display unit 230, receives various parameters which the 

. ^ ^ .jj user inputs to control the X-ray imaging. The operation 

elapsed time from the beginning of movement of a gnd and display unit 230 with the display or the touch screen thereof 

a position ot the gnd. displays imaging operation items input via the imaging 

FIG. 3 illustrates the interrelationship between the grid operation unit 220. 

and a sensor. When the user presses the X-ray exposure button 210, the 

FIG. 4 illustrates the construction of a grid position above-described digital radiographic apparatus directs the X 

measurement unit. 25 j-ays to the object 120 from the X-ray emitter 110 in response 

FIG. 5 is a timing diagram illustrating timings of the to a command of the imaging controller 180. The X rays, 

movement of the grid, operation of the sensor, and irradi- emitted from the X-ray emitter 110 and transmitted through 

ating X rays, Ihe object 120, enter the sensor 140 through the moving grid 

FIG. 6 is a graph for determining an exposure time of the 

X rays. The X rays incident on the sensor 140 are detected by the 

FIG.' 7 is a graph plotting the interrelationship of standard ^'"y ^"8"^ acquisi tion unit 150 in the sensor 140 in the 

X-ray exposure time, minimum X-ray exposure time, and "^"^ ^^"'^'"'^'^ ^'T^' T"^ ™fS« acquisition 

maximum X-ray exposure time. ™' com&r\s. the electrical signal into a digital 

r^^c OA •.. . . i^ signal. The X-ray image in the digital form is then fed to the 

HGS. 8A-8C illustrate operation screens of an operation 35 ■ ggo, through the imaging controller 180. 

display unit. t,, . JZ,. . . . 

^ The image processor 190 improves image quahty by 

FIG. 9 is a flow diagram illustrating an operation in which subjecting the X-ray image to a predetermined image 

a gnd table is automaticaUy adjusted when a set grid table processing, thereby displaying the X-ray image on the image 

fails to match an actual exposure time. display unit 200 or storing the X-ray image in the image 

FIG. 10 plots an example of a frequency distribution. storage unit 240 or both. 

DESCRIPTION OF THE PREFERRED During X-ray imaging, the imaging controller 180 con- 

tiK>ii5r\nTA>fcxrrc ols the grid movement controller 170 and the sensor 140 on 

EMBODIMENTS *u u • * u- u.u v 

the basis or the timing at which the X-ray emitter 110 expose 

One embodiment of the present invention is now the object 120 to the X rays. In this way, the timing of the 

discussed, referring to the drawings. irradiation of the X rays from the X-ray emitter 110 matches 

FIG. 1 is a block diagram showing a digital radiographic the timing of the movement of the grid 130 and the driving 

apparatus in accordance with one embodiment of the present of the sensor 140. 

invention. Discussed in greater detail below is the timing of the 

Referring to FIG. 1, an X-ray emitter 110, i.e., an X-ray 50 imaging controller 180 for controlling the driving of the grid 

source, emits and directs X rays to an object 120 during movement controller 170 and the sensor 140, and the grid 

X-ray imaging. A grid 130 removes scattered X rays that movement controller 170 for controlling the movement of 

have been generated within the object 120 which scatters the grid 130. 

X-ray emitted from the X-ray emitter 110. The grid movement controller 170 for controlling the 

A sensor 140 senses the X rays transmitted through the 55 movement of the grid 130 is discussed below, 

object 120 to obtain an X-ray image (a radiograph) of the Referring to FIG. 4, the grid movement controller 170 

object 120, and includes an X-ray image acquisition unit 150 moves the grid 130 using a link mechanism that converts a 

and an X-ray timing acquisition unit 160. The X-ray image rotary motion of a pulse motor (shown in FIG. 4) into a 

acquisition unit 150 detects the X rays transmitted through linearly reciprocating motion (shown in FIG. 2). 

the object 120, converts the X rays into an electrical signal 60 FIG. 2 is a graph ploUing the interrelationship between an 

in accordance with the intensity of the X rays, and further elapsed time from the beginning of movement of the grid 

converts the electrical signal from analog to digital to obtain 130 and the position of the grid 130 relative to the sensor 

an X-ray image. The X-ray timing acquisition unit 160 140. The abscissa represents the elapsed time from the 

detects the X rays emiued from the X-ray emitter 110 and beginning of the movement, while the ordinate represents 

transmitted through the object 120 on a time division basis. 65 the position of the grid 130 relative to the sensor 140. 

A grid movement controller 170 includes a grid position FIG. 3 illustrates the position of the grid 130 of FIG. 2 

measurement unit 175. In response to the results of a with respect to the sensor 140, viewed from above the grid 
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130 and the sensor 140. Referring to FIG. 3, blocks identical effect of the grid 130. Since the movement speed of the grid 

to those described with reference to FIG. 1 are designated 130 is the fastest in the vicinity of the zero point along the 

with the same reference numerals, and the discussion thereof path of the grid 130, contrast of the striped pattern due to the 

is omitted. grid 130 is minimized in the resulting X-ray image. 

Referring to FIG. 3, CLl designates the center line of the 5 Exposure of the object 120 to the X rays now starts with 

X-ray image acquisition unit 150 in the sensor 140, and CL2 the grid 130 moving slowly in the vicinity of the start point 

designates the center line of the grid 130. Dl is the distance of the movement (at 30 mm ) as shown in FIG. 2, In this 

between the center line CLl and the center line CL2. The case, the object 120 can be irradiated with X rays for a 

position of the grid 130 with respect to the sensor 140 is relatively long time from the start of the movement of the 
regarded as having a positive value when the center line CL2 lo grid 130 to the turning point thereof, 
of the grid 130 is located to the right of the center line CLl The grid movement controller 170 includes the grid 

of the X-ray image acquisition unit 150 (i.e., with the center position measurement unit 175 to measure the position of 

hne CL2 below the center line CLl in FIG. 3) if the X-ray the grid 130 in the movement thereof as discussed above, 
image acquisition unit 150 is viewed from the unshown FIG. 4 shows an example of the grid position measure- 

X-ray emitter 110, The position of the grid 130 with respect 15 ment unit 175, Referring to FIG. 4, there are shown the grid 

to the sensor 140 is regarded as having a negative value 130, and a link mechanism 410 which converts the rotary 

when the center line CL2 of the grid 130 is located to the left motion of the pulse motor into a reciprocating movement of 

of the center line CLl of the X-ray image acquisition unit the grid 130. A notch 420 is formed in a rotary disk 430. 
150 (i.e., with the center line CL2 above the center line CLl xhe rotary disk 430 having the notch 420 is attached to the 
m FIG. 3). The position of the grid 130 with respective to the 20 ^^^^^^ ^^^^^ ^^^^ ^^^^^ ^^..^.^^ ^ 

sensor 140 shown in FIG. 3 has a negative value (of -30 initial position of the pulse motor is determined by detecting 

™°^)- light, emitted by an unshown LED (Light Emitting Diode), 

In the interrelationship between the elapsed time from the through the notch 420 using a photosensor such as a pho- 

beginning of the movement of the grid 130 and the position todiode. When the grid 130 starts moving, the grid position 

of the grid 130 with respect to the sensor 140 as shown in measurement unit 175 counts the number of pulses that are 

FIG. 2, the grid 130 starts moving from a point 30 mm right fed to drive the pulse motor, thereby determining the angle 

of the center of the sensor 140 (i.e., at +30 mm) at the start of rotation of the pulse motor, and thereby determining the 

of movement, namely, at time zero (at the beginning of position of the grid 130. 

rotation of a pulse motor when the pulse motor is used to The grid position measurement unit 175 shown in FIG. 4 
drive the grid 130). The grid 130 moves to a position of a is an exemplary construction. The grid position measure- 
value that is determined by equation of 30 cos (cot) (w is an ment unit 175 is not limited to the exemplary construction 
angular speed of the pulse motor), and reaches to a point 30 shown in FIG. 4. Alternatively, the position of the grid 130 
mm left of the center line CLl of the sensor 140 (i.e., at -30 itself may be measured, and the results of measurement may 
mm) at time 200 ms. The grid 130 returns toward the center be fed to the grid movement controller 170. 
line CLl of the sensor 140 from the point 30 mm left thereof, ^^^^ ^^^^^^ ^ ^^^^^ ^.^^ pressing of the X-ray 
and then reaches to the point 30 mm right of the center line exposure button 210 to the actual exposure of the object 120 
CLl of the sensor 140 (namely, at the start point) at time 400 x rays from the X-ray emitter 110 (an X-ray generator, for 

example). To match the movement of the grid 130 and the 

The grid 130 moves reciprocatingly in front of the sensor driving of the sensor 140 with the irradiation of X rays from 
140 as shown in FIG. 2. In this case, the grid 130 momen- the X-ray emitter 110 in timing, a delay time needs to be 
tarily stops and resumes a turn movement at the end of the measured beforehand from the pressing of the X-ray expo- 
reciprocating movement (for example, at the point 30 mm sure button 210 (more precisely, from the supply of an X-ray 
left of the center Hne CLl of the sensor 140 (i.e., at -30 exposure start signal to the X-ray emitter 110) to the actual 
^Toa)), irradiation of X rays from the X-ray emitter 110. 

When the grid 130 turns in this way, the grid 130 The X-ray timing acquisition unit 160 in the sensor 140 is 

momentarily stops moving. In case that the X rays irradiat- used to measure the delay time. The X-ray timing acquisi- 

ing the object 120 and transmitted through the object 120 tion unit 160 has the function of chronologically or succes- 

enter the sensor 140 when the grid 130 momentarily comes sively measuring the intensity of X rays emitted from the 

to a halt, a striped pattern attributable to the grid 130 X-ray emitter 110 subsequent to the pressing of the X-ray 

distinctly appears in the resulting X-ray image. In other exposure button 210. 

words, the moire pattern due to the grid 130 becomes Referring to FIG. 5, the timings of the movement of the 

pronounced in the resulting X-ray image obtained by the gnd 130, the driving of the sensor 140, and the irradiation of 

X-ray image acquisition unit 150. x rays from the X-ray emitter 110 are discussed in greater 

To make the striped pattern attributable to the grid 130 55 detail below, 
less visible in the resulting X-ray image, the grid 130 needs FIG. 5 is a timing diagram showing the timings of the 

to be controlled such that exposure of the object 120 to the movement of the grid 130, the driving of the sensor 140, and 

X rays ends before the grid 130 reaches a turning point the irradiation of X rays. 

^^^^^o^' Referring to FIG. 5, the X-ray exposure button 210 is 

The position of the grid 130 with respect to the sensor 140 eo pressed to begin X-ray exposure at time Tj. The grid 

at the start of the X-ray exposure will now be discussed. movement controller 170 controls the grid 130 at time T2 

Now the object 120 is irradiated with X-ray with the grid after a predetermined time elapsed from time Tj to move the 

130 passing the zero point as shown in FIG. 2 (for a grid 130. The predetermined time is determined so that the 

predetermined duration of time with the grid 130 being in effective duration of time of the moving grid (corresponding 
the vicinity of the zero point) when the grid 130 moves fast, 65 to an effective area) shown in FIG. 2 and FIG. 4 includes a 

The intensity distribution of the X rays obtained by the duration of time (assumed exposure time) during which the 

X-ray image acquisition unit 150 is less subject to the cutoff X rays are irradiated from the X-ray emitter 110. 
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At time T3, the X-ray image acquisition unit 150 starts an By vselecting (appropriate exposure time) appropriate one 

accumulation operation for accumulating the X ray signal of the durations of time corresponding to the movement 

transmitted through the object 120. The accumulation opera- speed of the grid 130 (i.e., the angular speed of the pulse 

tion lasts until the intensity of the X rays acquired by the motor), a grid table is formed and stored in the imaging 
X-ray timing acquisition unit 160 becomes zero. At time T3, 5 controller 180. The grid table is displayed on the operation 

the X-ray exposure start signal is fed to the X-ray emitter display unit 230 on the imaging operation unit 220 so that an 

110, and the X-ray emitter 110 begins generating X rays. The ^f the digital radiographic apparatus may 

X-ray emitter 110 has a delay time DT, characteristic of the utilize the grid table 

X-ray generator, from the supply of the X-ray exposure start ^ ./^ , 1 , , . ^ , , , , 
signal to the actual irradiation of X rays. The delay time DT Specifically, the grid table is formed and displayed in the 
is measured (beforehand) using the X-ray liming acquisition ^^^^ T^^T^T^ m accordance with the movement speed 
unit 160, and then the start time of the accumulation S^id 130 (or the angular speed of the step motor), 
operation by the X-ray image acquisition unit 150 is deter- wherein T^ represents a minimum X-ray exposure time and 
mined (i.e., the start time of the accumulation operation is represents a maximum X-ray exposure time. The mini- 
delayed by a predetermined time from the supply time of the ^"^^y exposure time T^ is determined by subtracting 
X-ray exposure start signal with the delay time DT the value of TBI from the value of TCI for each movement 
accounted for) speed of the grid 130, The maximum X-ray exposure time 

Hie X-ray timing acquisition unit 160 measures the delay determined by subtracting the value of TBI from the 

time DT If the X-ray image acquisition unit 150 has the ^^^^^ ^^^^ movement speed of the grid 130. 

function of successively measuring the intensity of X rays, As described above, the digital radiographic apparatus 

the X-ray image acquisition unit 150 may measure the delay tends to suffer from a degradation in the image quality of the 

time without using the X-ray timing acquisition unit 160. ^-ray image attributable to the moire pattern caused by the 

Alternatively, the X-ray timing acquisition unit 160 may be grid. Therefore, in X-ray imaging, the movement of the grid 

arranged in the X-ray emitter 110 rather than in the sensor 130 needs to be precisely controlled. A time interval of the 

140. grid table T^^T^T^ is shorter than that permitted in the 

FIG. 6 is a graph for determining the X-ray exposure time conventional film/screen system. Aplurality of grid tables is 

duration in case that the grid 130 reciprocatingly moves in ^^^^ prepared to permit a wide range of choice in the grid 

front of the sensor 140. The abscissa represents the elapsed ^^^^^^ during X-ray imaging. 

time from the beginning of the movement of the grid 130, For example, when the chest of a human body is X-ray 
while the ordinate represents the angular speed of the pulse 3^ imaged, the X-ray exposure time is set to be as short as 10 

motor corresponding to the movement speed of the grid 130. ms so that a heart and the peripheral portion thereof are not 

As shown, TAl represents the time from when the grid blurred in the X ray image regardless of the heart beat. 

130 starts moving until when the grid 130 reaches the However, there are times when the X-ray exposure time 

turning point. Specifically, TAl indicates a reference becomes as long as one second because the transmittance of 
intended to stop the exposure of the object 120 to the X rays 35 the X rays through the abdomen and the like of the human 

before the turning point of the grid to reduce the contrast of is typically low. To radiograph the human body, the digital 

the striped pattern due to the grid in the resulting X-ray radiographic apparatus must work with the exposure time 

image acquired by the X-ray image acquisition unit 150. ranging from several ms to several hundred ms or several 

TBI represents the time from when the grid 130 starts seconds. Thus, a plurality of grid tables (for example, four 
moving untU when the X-ray emitter 110 starts directing X 40 °^ ^^^^ ^rid tables) are required. 

rays to the object 120, i.e., the time fi-om when the grid 130 However, the use of multiple grid tables provided is 

starts moving until the grid 130 reaches a position where the accompanied by an inconvenience. The operator of the 

startoftheirradiationof X rays is appropriate. TCI repre- digital radiographic apparatus has difficulty in properly 

sents the time from when the grid 130 starts moving until the selecting the grid table appropriate for each X ray imaging 
moire pattern due to the grid is reduced to predetermined 45 application. In accordance with this embodiment, an imag- 

degree subsequent to the start of the irradiation of X rays. iog button corresponding to each region of the body to be 

Specifically, the duration of time from TBI to TCI is the imaged is arranged in the imaging operation unit 220, and is 

time needed for the required number of lead foil lines takes associated with the respective grid table. When the operator 

to pass by a single pixel of the X-ray image acquisition unit presses one imaging button, the grid table corresponding 
150 from the beginning of the irradiation of X rays to reduce 50 thereto is automatically selected. Based on the selected grid 

the moire pattern due to the grid to predetermined degree. table, the grid movement controller 170 controls the move- 

The number of passing lead foil lines of the grid 130 is ^^^^ ^^e grid 130. 

obtained by dividing the product of the movement speed of In the imaging method that is set by an imaging button 

the grid and the irradiation time of the X rays (i.e., the time corresponding to each region of the body, a predetermined 
integral of the grid movement speed, namely, the distance of 55 grid table chosen based on a standard imaging time (an 

travel of the grid) by the pitch of the lead foil of the grid 130. X-ray exposure time) is selected. 

When the movement speed of the grid 130 and the pitch of Recently, imaging information (imaging request or order- 

the lead foils of the grid 130 are determined, a required ing information) is transmitted and received through an 

X-ray exposure time to reduce the moire pattern to prede- HIS/RIS (Hospital Information System/Radiology Informa- 
termined degree is calculated from the required number of 60 tion System) system, and information for each region of the 

passing lead foil lines of the grid 130. The X-ray image object (information concerning a region to be imaged) 

acquisition unit 150 thus acquires the X-ray image free from included in the imaging information is associated with a 

the moire pattern by causing the X-ray emitter 110 to start predetermined grid table. When the imaging information is 

the exposure of the object 120 to the X rays from TBI, and received, the imaging controller 180 automatically selects a 
to end the exposure of the object 120 to the X rays for 65 grid table matching the imaging information, and designates 

somewhere between any of durations 610-1, 610-2, 610-3, the movement speed of the grid 130 based on the selected 

and 610-4 (i.e., between TCI and TAl) as shown in FIG. 6. grid table. The grid movement controller 170 controls the 
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movement of the grid 130 in accordance with the designated grid 130. Thus, the grid movement controller 170 properly 

movement speed of the grid 130, thereby appropriately controls the movement of the grid 130. 

moving the grid 130. FIGS. 8A-8C illustrate example operation screens pre- 

When the movement speed of the grid 130 is continuously sented on the operation display unit 230 in the imaging 
varied, the minimum X-ray exposure time and the maxi- 5 operation unit 220. 

mum X-ray exposure time in the grid table formed in Referring to FIG. 8A, a touch panel 810 includes a 
accordance with FIG. 6 are substantially also continuously liquid-crystal display and an analog resistive-film touch 
varied. A grid table T^^T^T^ thus selected is more appro- censor sheet. A mouse 815 is used to move a pointer on the 
priate for the imaging region than one grid table which is ^^&y ^^^^f, ^V^^"^ the display screen, 
selected from a plurality of grid tables T^^T^T^ that are lO Designated as 820 are cables for power and control lines, 
formed in response to the discretely selected movement . The touch panel 810 presents a display of imaging opera- 
speeds of the end 130 tional parameters. The imaging operational parameters are 

TTTL I .J . . . 1 ,1 selected and input by either the touch panel 810 or the mouse 

When the number of gnd tables is mcreased to allow a ^ combination thereof. The cables 820 include a 

grid table appropriate for an imagmg region to be power cable, a VGA (Video Graphics Array) cable, a touch 

designated, a standard exposure time t is associated with the p^nel controlling serial cable, and a mouse serial cable, 

imaging region so that the operator easily selects an opti- ^ jj^age display area 825 displays a captured X-ray 

mum grid table. image. An object information display area 830 displays 

When the imaging button corresponding to the imaging information relating to the object. An imaging method object 
region is pressed by the operator, or when the imaging parameter display area 835 displays parameters of an iraag- 
information transmitted through the HIS/RIS system is ing method object. An imaging method object display area 
received, the imaging controller 180 selects the standard 840 displays an imaging method object in response to a 
exposure time t in response to the imaging button or the status of the apparatus. A message display area 845 displays 
information concerning the imaging region in the imaging status and messages of the apparatus or a system. A modi- 
information, and then selects the grid table T^^T^T^ based fication button 850 is used in case that imaging conditions 
on the standard exposure time I. The grid movement con- and/or image processing parameters are modified. 
troUer 170 thus controls the movement of the grid 130 in When X-ray imaging, the operator selects a desired imag- 
accordance with a grid table appropriate for the imaging ing method object from among the imaging method objects 
region. displayed in the imaging method object display area 840. 

FIG. 7 is a graph plotting the interrelationship of the The selection of the imaging method object may be canried 

standard X-ray exposure time t, the minimum X-ray expo- out by pressing the imaging method object displayed on the 

sure time T^, and the maximum X-ray exposure time Tjr. touch panel 810 or by sending the imaging information 

As shown, the abscissa represents the elapsed time (the through the HIS/RIS system. 

X-ray exposure time) from when the X-ray emitter 110 has In the imaging method object, a number of imaging 

started irradiating the object 120 with the X rays, and the method parameters and standard imaging conditions corre- 

ordinate represents the movement speed of the grid 130. TA2 spending to an imaging region are set. Since the standard 

is a curve representing the maximum X-ray exposure time exposure time t under the standard imaging conditions is 

T^, namely, the value that is obtained by subtracting the associated with the grid table in one-to-one conrespondence 

value of TBI from the value of TAl shown in FIG. 6. TC2 as already discussed with reference to FIG. 7, the grid table 

is a curve representing the minimum X-ray exposure time is also set in the imaging method object. Specifically, when 

T9, namely, the value that is obtained by subtracting the the operator selects the imaging method object, the standard 

value TBI from the value of TCI shown in FIG. 6. imaging time (the standard exposure time) t set beforehand 

Each of the curves TA2 and TC2 is plotted against the corresponding to the selected imaging method object is 

X-ray exposure time along the abscissa on a one-to-one displayed, and the grid table T^^T^Tjc responsive to the 
correspondence basis. Accordingly, the minimum X-ray 45 standard imaging time t is thus selected, 

exposure time T^ and the maximum X-ray exposure time T^ When the X-ray exposure button 210 is pressed with a 

can be a function of the standard exposure time t, and are "Ready" message displayed on the message display area 

thus respectively expressed as the minimum X-ray exposure 845, the X-ray emitter 110 directs the X rays to the object 

time T5(t) and the maximum X-ray exposure time 120, thereby X-ray imaging of the object 120 is performed. 

The standard exposure time t is set to subdivide the time 50 During X-ray imaging, the grid movement controller 170 

interval between the minimum X-ray exposure time T^(t) moves the grid 130 as shown in FIGS. 2, 3, and 5. 

and the maximum X-ray exposure time Tjr(i) by a ratio of When the object 120 is X-ray imaged in this way, the 

m:n (m and n are natural numbers) to tolerate a deviation of captured X-ray image is displayed on the image display area 

an actual exposure time from the standard exposure time t 825 as shown in FIG. 8B. When the modification button 850 
depending on a difference in the thickness of the object. 55 is pressed with the image method object selected as shown 

For example, TD2 shown in FIG. 7 indicates the standard in FIG. 8B, the display presented by the touch panel 810 

exposure time t that is obtained by subdividing the time changes to the one shown in FIG. 8C, 

interval between the minimum X-ray exposure time T^(t) Referring to FIG. 8C, the touch panel 810 displays 

and the maximum X-ray exposure time Tjrii) by a ratio of parameters required in X-ray imaging and sets those param- 
5:2. 60 eters corresponding to the selected imaging method object. 

When the operator selects the imaging region, the stan- The displayed parameters include exposure time, i.e., the 

dard exposure time t corresponding to the imaging region is standard exposure time. By modifying the exposure time 

selected using the interrelationship of the standard X-ray using an up button or a down button, a grid table corre- 

exposure time t, the minimum X-ray exposure time T^, and spending to a modified exposure time is displayed on the top 
the maximum X-ray exposure time T^ shown in FIG. 7. Also 65 portion of the screen. 

selected are the grid table T^^T^Tg; responsive to the In X-ray imaging, a function of adjusting a dose of X-ray 

standard exposure time t, and the movement speed of the irradiating the object, called an AEC (Auto Exposure 
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Control) function, is available. However, X-ray imaging equal to or more than M. When the total frequency is equal 

must be performed with an exposure time matching a grid to or more than M, data transnormal in the frequency 

table to obtain an X-ray image free from a moire pattern due distribution is deleted (for example, data of X-ray exposure 

to the grid. Further, as described above, a grid table corre- time having the lowest frequency is deleted). In step S940, 
spending to the imaging method object needs to be set. 5 the imaging controller 180 determines whether the currently 

Therefore, if an actual exposure time adjusted through the set grid table ^T^T^T^T^; is appropriate for the frequency 

AEC fails to match the grid table (the exposure time defined distribution formed by the current imaging cycle. For 

by the grid table), the grid table needs to be modified to example, in this determination, the imaging controller 180 

match the actual exposure time. references a difference between an expected value of the 

The operation of automatic adjustment of the grid table ^0 firequency distribution of the X-ray exposure time and the 

performed when the actual exposure time fails to match the standard exposure time t shown in FIG. 7 currently set in the 

grid table is discussed below, grid table, or performs a determination based on whether at 

FIG. 9 is a flow diagram showing the operation of the least 80% elements of the fi^quenc^^^ 

automatic adjustment of the grid table performed when the ^*°Se of the gnd table Js^T-fT-E- FIG. 10 shows an 

actual exposure time fails to match the grid table. This example of a firequency distnbution. 
operation is performed by the imaging controller 180. When it is determined in step S940 that the frequency 

When one imaging cycle is completed with the imaging distribution of the X-ray exposure time fails to satisfy the 

method object selected, an actual exposure time T;, of the ^^"^^ ^^^^^^^^^^ cL? o^'^^' T^^T^T^T^, the 

object 120 to the X rays is acquired from the X-ray emitter P^f^^^^ proceeds to step S945. The imaging controller 180 

110 or the X-ray timing acquisition unit 160 in step S905, f'""^ ] ^nd table that meets the formed frequency distn- 
and is stored as data. In step S910, the X-ray exposure time ^^/^^^ selection the imaging controller 180 selects 

T;, acquired in step S905 is grouped according to intervals I ^^^^'^^V^f f that an expected value of the 

that are predetermined by discretely subdividing the X-ray frequency distnbution of the X-ray exposure time substan- 
exposure time, and a frequency distribution of the X-ray „ t^al y comcides >.nth the standard exposur^^^^^^ 

exposure lime is formed (updated). Specifically, one piece of [^^^^ ^}^\^^ 1^^^ ^[^^^^^ 80% elements of the 

data (a frequency of one) is added to the frequency distri- ^'"1^"^^^ distribution satisfies the range of the grid table 

bution of the X-ray exposure time formed at a preceding I5- 1 - 

imaging cycle, in accordance with those discrete intervals. ^^^^ ^^ew grid table is selected in step S945, the 

In step S915, the imaging controller 180 determines 30 "^^ly selected grid table is ^ 
whether the number of imaging cycles, including this imag- ^^^P P^°^^^ 

ing cycle, is equal to or greater than N with the same When it is determined m step S940 that the frequency 

imaging method object selected. distnbution of the X-ray exposure time satisfies the range of 

• J * " • J * *u 1 c ■ ' 1 Ihe currently set grid table To^T^T=-, the imaging control- 
When it is determined that the number of imagmg cycles 1 ion J *u * J -r • *u -J * 1 1 /• 

• 1 xr *u • » 11 1 0A J . • *u * l^r 180 ends the process without modifying the gnd table (m 
IS less than N, the imaging controller 180 determines that a 35 sQlO'k / & & v 

correct (appropriate) grid table can not be presumed from ^ ,1 , , i- 

the frequency distribution based on the past X-ray exposure , ^ discussed above, while the X-ray emitter 110 directs 

time. The frequency distribution is not used, but the acquired ^^^^^ ^^^^ ^^e object 120, the gnd movement controller 

X-ray exposure time T;^ is used. ^'^^ ^^"^^^^ movement of the gnd 130 so that the 
T * omn *u • • * 11 IDA J . • An reciprocatiugly moviug grid 130 moves in One direction with 
In step S920, he imaging controller 180 determines 40 ^ p^obabflity. The X-ray image acquisition unit 150 

whether the currently acquired X-ray exposure time falls ^ rays transmitted through the object 120 and the 

wi hm the range of the grid table T,|T^T,. When it is .^^^ ^ ^ ^ 

determmed that the currently acquired X-ray exposure time iniensitv of the detected X raw 

T, falls outside the range of the grid table T^ST^T^, the ""*n«'y °' t^'^'^^ ^ '^^y^- 

process proceeds to step S925. The imaging controller 180 45 Accordm^y, the probability that when the gnd 130 turns 

newly selects a grid table T^ST^TST^ within which the and moves the object WO ^stiU exposed to the X rays f^^^ 
X-ray exposure time falls. This selection may be made so X-ray emitter 110 is reduced. In this way, the probability 

that the standard exposure time t discussed with reference to the X-ray image acquisition umt 150 presents an X-ray 

FIG. 7 equals the X-ray exposure time T;,. or so that a ''^^"S ^^^^ ""^ °° ""^''^ P^"«™ ^""^ '° gnd 130 is 

modification from the grid table TySTST^ for the current 50 heightened. 

imaging is minimized The X-ray timing acquisition unit 160 measures the actual 

When the new grid table is selected in step S92S, the f °^ ''T^ which the object 120 is exposed to 

imaging controller 180 sets again the newly selected grid ^ .^"^l"^^ °" 

♦ ui * *u • * *u J u- * • 4. oHffA J J measured time, the conneuration is provided that the end 
table to the imaging method object in step S950, and ends ' , ^-^^ . ^ , , . , 
the process movement controller 170 controls the movement of the gnd 

„ . . , . , . « ^ to increase the probability that the grid 130 moves in 

When it is determmed in step S920 that the currently direction during X-ray exposure (irradiation) time. In 

acquired X-ray exposure time T;^ falls within the range of the ^his way, the grid 130 is controlled at an appropriate speed 
gnd table T^^T^T^, the process ends without modifying timing. Thus the probability that the X-ray image 

the gnd table (m step S930). acquisition unit 150 provides an X-ray image with less or no 

When it is determined in step S915 that the number of moire pattern is heightened, 
imaging cycles is equal to or greater than N, the imaging A standard time for exposing the object 120 to the X rays 

controller 180 proceeds the process to step S935 assuming and/or an ejffective time duration including the standard 

that a correct grid table can be presumed from the frequency exposure time are set as an X-ray exposure time. Parameters, 
distribution based on the past X-ray exposure time. 55 s^ch as the grid movement speed that is determined based on 

The imaging controller 180 determines in step S935 the X-ray exposure time, are set as the grid table for 

whether the total frequency of the frequency distribution is controlling the movement of the grid 130. The grid move- 
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ment controller 170 thus controls the movement of the grid 
130 in accordance with the grid table. By simply moving the 
grid 130 in accordance with the grid table, the probability 
that the reciprocatingly moving grid 130 moves in one 
direction during X-ray irradiation and that the X-ray image 
acquisition unit 150 acquires an X-ray image having less or 
no moire pattern due to the grid 130 is heightened. 

Since the grid table used in X-ray imaging is displayed on 
the operation display unit 230, the operator judges from the 
actual X-ray exposure time whether an image with less or no 
moire pattern due to the grid 130 has been obtained. When 
the operator sets X-ray exposure time beforehand and radio- 
graphs the object 120, the operator has a chance, by the 
displayed grid table, to set an X-ray exposure time with 
which the probability that an image with less or no moire 
pattern due to the grid 130 is obtained is heightened. 

Since a grid table is preset for each imaging method object 
displayed on the imaging operation unit 220, the operator 
easily selects the grid table that is preset appropriately for 
radiographing each region of an object, by simply selecting 
the imaging method object. This arrangement saves the labor 
time an operator may take to consider and then select the 
grid table at each imaging cycle. 

The standard exposure time t is set to subdivide the time 
interval between the minimum X-ray exposure time T^ and 
the maximum X-ray exposure time T^; by a ratio of m:n (m 
and n are natural numbers), and the grid table is selected 
based on the standard exposure time t. When presetting the 
grid table for the imaging method object, the operator easily 
presets the grid table using the standard exposure time t as 
a guideline. 

If the grid table set in each imaging method object is 
determined to fail to satisfy the actual X-ray exposure time 
(from statistics of past X-ray exposure times), the grid table 
satisfying the X-ray exposure time during imaging is auto- 
matically selected. The selected grid table is again set in the 
imaging method object. Even if a proper grid table is not set 
in one of score of imaging types, a grid table, which may be 
inappropriate, is replaced with another appropriate grid table 
which is considered more appropriate from the statistics of 
past X-ray exposure times. 

The present invention has been discussed in connection 
with the digital radiographic apparatus using X-ray. 
However, the present invention is not limited to apparatuses 
using X rays. The present invention is applicable to appa- 
ratuses which obtain object images using other radiation 
transmitted by the object. 

Modifications of the Embodiment of the Present Invention 

A program code of software for carrying out the functions 
of the invention can be loaded into a computer in a system 
or an apparatus connected to a variety of devices so that the 
devices perform the functions of the invention. The variety 
of devices operate in accordance with the program stored in 
the computer (CPU or MPU) in the system or apparatus. The 
functions and process steps of the invention are thus carried 
out. Such embodiments fall within the scope of the present 
invention. 

The program code of software performs the functions and 
the process steps of the invention. The program code itself, 
and means for feeding the program code to the computer, for 
example, a storage medium for storing the program code, 
fall within the scope of the present invention. 

Available as storage media for storing the program code 
are a floppy disk, a hard disk, an optical disk, a magneto- 
optical disk, a CD-ROM, a magnetic tape, a nonvolatile 
memory card, ROM and the like. 
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By executing the program code read by the computer, the 
computer realizes the functions or performs the process 
steps of the invention. Furthermore, the functions or the 
process steps of the invention are realized or performed in 
cooperation with the OS (operating system) running on the 
computer or another application software program accord- 
ing to the instruction of the program code. Such a program 
code falls within the scope of the present invention. 

The program code is read into a memory incorporated in 
a function expansion board in the computer or in a function 
expansion unit connected to the computer. The CPU 
mounted on the function expansion board or the function 
expansion unit performs partly or entirely the actual process 
in response to the instruction from the program code. The 
functions or the process steps of the invention are realized or 
executed through the process. Such a program code falls 
within the scope of the present invention. 

The program code is delivered in a computer-readable 
storage medium. The program code may also be delivered 
on-line through a communication network such as the Inter- 
net. 

While the present invention has been described with 
reference to what are presently considered to be the pre- 
ferred embodiments, it is to be understood that the invention 
is not limited to the disclosed embodiments. To the contrary, 
the invention is intended to cover various modifications and 
equivalent arrangements included within the spirit and scope 
of the appended claims. The scope of the following claims 
is to be accorded the broadest interpretation so as to encom- 
pass all such modifications and equivalent structures and 
functions. 

What is claimed is: 

1. An apparatus for radiographing an object, comprising: 

an X-ray radiation unit for radiating X-ray; 

a grid arranged in an X-ray radiation path; 

a grid movement controller for changing a movement 
speed of the grid by changing a turn speed of a motor, 
comprising a link mechanism for changing a turn 
movement of the motor into a straight movement of the 
grid; 

a sensor unit for converting the X-ray into image data; 

an input unit for inputting information relating to a region 
of a body; and 

an imaging controller for controlling (i) the time for the 
X-ray radiation unit to start radiating the X-ray, (ii) the 
time for the grid movement controller to start rotating 
the motor, and (iii) the time for the sensor unit to start 
storage, by associating one with another, 

wherein the imaging controller (a) selects a standard 
radiation exposure time and the turn speed based on the 
information input into the input unit, (b) controls the 
radiation exposure starting time of the X-ray radiation 
unit based on the selection, and (c) causes the grid 
movement controller to rotate the motor at the turn 
speed, 

wherein the standard radiation exposure time is selected 
based on the maximum X-ray radiation time to be 
determined according to the region of the body, and 

wherein the imaging controller is configured such that it 
controls the radiation exposure starting time of the 
X-ray radiation unit so that the standard radiation 
exposure time will be y divided by a ratio of m:n, where 
y is the time interval between the minimum X-ray 
radiation time and the maximum X-ray radiation time, 
m and n are namral numbers, and the minimum X-ray 
radiation time and the maximum X-ray radiation time 
are determined according to the region of the body. 
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2. An apparatus according to claim 1, wherein the mini- 
mum X-ray radiation time is the time from when radiation 
starts until when the grid moves a predetermined distance, 
and the maximum X-ray radiation time is the time from 
when radiation starts until when the grid starts a turn 
movement. 

3. An apparatus according to claim 2, wherein the prede- 
termined distance is determined so that the value of the 
predetermined distance multiplied by a pitch of a lead foil of 
the grid will be a predetermined value. 
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4. An apparatus according to claim 1, further comprising 
a display unit for displaying one or more combinations of the 
minimum X-ray radiation time and the maximum X-ray 
radiation time, and the standard radiation exposure time. 

5. An apparatus according to claim 1, wherein m is 5 and 
n is 2, and wherein the standard radiation exposure time is 
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[57] ABSTRACT 

A method and apparatus for establishing an always- visible 
class of windows in a computer-implemented windowing 
environment is provided, A user may designate one or more 
windows as always-visible windows. If an always-visible 
window overlaps with a non-always-visiblc window, then 
the always-visible window is displayed on top of the non- 
always-visible window. Always-visible windows are pre- 
vented from overlapping with each other. Techniques are 
provided for implementing the always-visible window class 
in a manner that complies with the X Windows system. 
AccOTding to one technique, the override redirect attribute is 
used as a flag to designate which windows are always- 
visible windows. According to an alternative technique, a 
list of always-visible windows is maintained as a property 
attached to a root window. 
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WINDOW SYSTEM PREVENTING OVERLAP . the user. For example, PC Tools for Windows provides a 

OF MULTIPLE ALWAYS-VISIBLE WINDOWS window-based graphical desktop that aUows a user to 

tm^V OF I Hh INVENTION "always on top" window is assigned a position in the stack 

The present invention relates to window management on ^ is higher than the highest position assigned to 

computer systems, and more specifically, to a method and ordinary windows. Consequently, if any window is manipu- 

i^paratus for displaying a class of always-visible windows ^ such a way as to overlap with the "always on top" 

in an X Windows system. window, the overlapping portion of the "always on top" 

window will cover the other window in the common screen 

BACKGROUND OF THE INVENTION lo region. 

A window is a user interface object that establishes a This approach works well as long as there is only one 

correlation between a particular set of data and a particular window that contains cnicial information. However, a jarob- 

screen region. Unless the window is hidden or covered by lem arises when more than one "always on top" window is 

another user interface object, the set of data is typicaUy needed. In PC Tools fos: Windows, if two windows are 

displayed in the corresponding screen region. The use of designated as "always on top" windows, they behave the 

windows has proven to be an effective way of communi- same with respect to each other as normal windows do with 

eating information to a computer user. For example, a word respect to each other. That is, one "always on top" window 

processing system that allows a user to open multiple will have a higher stack order position than the other. When 

documents may provide a separate window fcH* displaying the two "always on top" windows are caused to overlap, the 

the contents of each of the open documents. higher-ordered window will cover the lower-ordered win- 

For added flexibility, many window management systems ^ conmaon sareen region. The information con- 
allow windows to overlap with each other. Two windows tained in the covered portion of the lower-ordered window 
ovciiap when there is an intersection between the screen longer visible to the user. Thus, when m<Mre than one 
regions associated with the windows. If the contents of two "always on top" window is displayed, there is no way to 
<» more overlapping windows arc displayed in the common ^ ensure that critical infomation will always be visible to the 
screen region, the information may appear garbled and 

difficult to read. Consequently, the window management One way to avoid this problem is to allow only one 

system must determine what should be displayed in the "always on top" window at a time. For example, Erench 

region that is common to overlapping windows- Patent Publication number 2,693,810 describes a window 

To address this problem raised by overlapping windows, management system that provides one window that cannot 

most window management systems assign each window a obscured by other windows. When a user designates a 

position in a "stack order*'. When multiple windows share second window as a non-obscurable window; the first non~ 

the same screen region, only the information from the obscurable window ceases to be non-obscurable. Because 

window that has the hi^er position in the stack order is 3^ ^his system does not support multiple non-obscurable 

shown in the screen region. Thus, the portion of the lower- windows, its utility is limited 

ordered window that corresponds to the common screen Based on the foregoing, it is clearly desirable to provide 

region will be "covered" by the portion of the higher-<M-dered a windowing system that allows information to be simulta- 

window that corresponds to the common screen region. To neously displayed in multiple always-visible windows, 

a user, this makes the lower-ordered window appear as if it ^ Further, it is desirable to provide an always-visible class of 

were physically below the higher-ordered window. windows in a window management system that is consistent 

Most window management systems allow users to move, with the current X-Windows standards. In addition, it is 

resize and change the stack order ofindividual windows. For desirable to provide a windows management system that 

example, in the System 7 operating system available from allows users to combine window attributes including an 

Apple Computer, Inc., a window assumes the highest posi- 45 always-visible attribute and a transparent background 

tion in the stack order when a user clicks on a portion of the attribute, 
window using an input device such as a mouse or trackbaU. 

A window is resized by dragging the bottom corner of the SUMMARY OF THE INVENTION 

window until the window has the desired dimensions. A A method and apparatus for establishing an always-visible 

window is moved by dragging the title bar of the window to 50 class of windows in a computer-implemented windowing 

a new position on the saeen display. environmcait is provided A user may designate one or more 

In some applications, it may be important to ensure that windows as always-visible windows. If an always-visible 

certain infcamation is always visible to the user. However, if window overlaps with a non-always-visible window, then 

the information is displayed in a window in a system that the always-visible window is displayed on top of the non- 

allows windows to overly, then the vital infonnation may 55 always-visible window. Always-visible windows are pre- 

become hidden from flie user. For example, the infoonation vented from overlapping with each other. Techniques are 

may become covered if the user performs some action that provided for implementing the always-visible window class 

would cause a window that is higlher in the stack order to in a manner that complies with the X Windows system, 

overiap with the given window. Numerous types of user According to one technique, the oveiride redirect attribute is 

actions can cause this situation to arise. For example, the 60 used as a flag to designate which windows are always- 

uscr may cause an akeady-ovcarlapping window to assume a visible windows. Acccs-ding to an alternative technique, a 

position in the stack order that is higher than the position of list of always- visible windows is maintained as a property 

the given window. Alternatively, the user may move a attached to a root window. 

window that is akeady higher in the stack order to a saeen According to one aspect of the invention, a method for 

position that overlaps with the given window. 65 displaying inf ramation on a display device of a computer 

Various attempts have been made to prevent important system is provided. According to the method, information is 

information displayed m a window from becoming hidden to simultaneously displayed in a plurality of windows on the 
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display device. The plurality of windows includes a first as to cause the always-visible to overlap with another 

always-visible window. A plurality of configurations that always-visible window; 

correspond to the plurality of windows is maintained. yIG. 9 illustrates the soeen of FIG. 2 after fee opearation 

Specifically, the configuration for each window is main- shown in HG. 8; 

tained to reflect the location and dimensions of the window 5 illustrates a functional block diagram of a system 

on the display device. tl^at implements always-visible windows according to an 

An event that causes a portion of a second window to embodiment of the invention; 
occupy a common region on the display device with a 11 illustrates a functional block diagram of a system 
portion of the first always-visible window is detected. Upon implements always-visible-windows acceding to an 
detecting such an event, it is detoinined wheth^^^ second lo ^^^^^^ embodiment of the invention; 
window is also an always-visible wmdow. If the second _ ^ ^ ^ ^ vr^- * ^• 
windowisanalways-visi^lewindow^thentheconfiguration ^ ^^G. 12a lUustrates he operaU^^^ 
of oneorbothof tiiewindowsisalteredtopreventoverlap ^o an embodment of the mvention; 
between the windows. If fee second window is not an FIG. 1^ iUuslrates steps for designating an always- 
always-visible window, feen the first window is displayed 15 visible window according to one enibodiment of fee inven- 
over the second window. *^^'*» 

According to another aspect ofthe invention, a stack order ^ iUustrates steps for designating an always- 

is maintained for the plurality of windows. User input is visible window according to an alternative embodiment of 

received which causes the second window to have a higher invention; 

position in fee stack order than fee first always-visible FIG. 13a illustrates fee operation of a bump window 

window. If fee second window is not an always-visible manager according to an embodiment of the invention; 

window, feen fee stack order is altered to cause fee first piQ, 13^ illustrates fee steps performed by fee bump 

always- visible window to have a higher position in fee stack window manager to initialize an always- visible window Hst; 

order fean fee second window. ^ j^q ifluslrates fee steps petformed by fee bmnp 

Various techniques may be used to designate a window as window manager to process fee always-visible window list 

an always-visible window. AccOTding to one embodiment, to avoid overlap between always-visible windows; 

fee override-redirect attribute provided in fee X-windows illustrates the steps performed by fee bump 

system is used as a flag to designate always-visible win- window manager in response to detection of an event which 

dows. According to anofeer embodiment, a property 3^ ^^^^^ ^ window; and 

attached to fee root window is defined as an always-visible . . . r *u « u mi * ^ • 

window list To designate a window as an always-visible ^^^^J^' ^ continuation of the flow chart illustrated m 
window* an entry feat identifies fee window is added to fee 

always-visible Hst Bofe ^fec«e techniques maybe DETAILED DESCRIFnON OF THE 

used wifeout deviatmg firom fee X-Wmdows standard. 35 PREFERRED EMBODIMENT 

BRIEF DESCRIPTION OF THE DRAWINGS ^ mefeod and apparatus for simultaneously displaying 

The present invention is illustrated by way of example, multiple always-visible windows is described. In fee fol- 

and not by way of limitation, in fee figures of fee accom- lowing description, for fee purposes of explanation, numer- 

panying drawings and in which like reference numerals refer 40 specific details are set forfe in order to provide a 

to similar elements and in which: thorough understanding of fee present invention. It wiU be 

FIG. 1 illustrates a computer system upon which fee apparent, however, to one skilled in fee art feat fee present 

preferred embodiment of fee present invention can be imple- invention may be practiced wifeout feese specific details. In 

mented' ^^^^ instances, well-known structures and devices are 

no. 2 fllustrates a display device that is simultaneously 45 ^howa in block to^amft«mm^^^ to avoid unnecessarily 

displaying two always-visible windows and two normal ° ^ 8 

windows on a screen according to an embodiment fee Referring to FIG. 1, fee computer system upon which fee 

invention* preferred embodiment of fee present invention can be imple- 

HG. 3 fllustrates the screen of HG. 2 afte a user has 100 Computer system 100 comi«ses a 

selected a noanal window that overlaps with an always- » !»lf <f communication means 101 for communicat^g 

visible window information, and a i»ocessing means 102 coupled with bus 

„„ , .„ ' , , . t • • 101 for procesdng infonnation. System 100 further com- 

FIG. 4 Illustrates the screen of HG. 2 when fte user is m ^^^^ ^ ^^^^^ ^^^^^^ ^ ^ „ther dynamic 

the process ot selecting and emaiging a normai wmdow to ^^^.^^ ^^ 

as main memory), coupled to 

cause the normal wmdow to overlap wife an always-visible infoimation and instructions to be 

window; executed by processor 102. Main memcxy 1<M also may be 

FIG. 5 iUustrates the screen of FIG. 2 when fee resize ug^d for storing temporary variables or ofeer intermediate 

operation illustrated in FIG. 4 has completed; information during execution of instructions by processor 

FIG. 6 illustrates fee screen of HG. 2 when a user has 102. Coir4)utcr system 10© also conqmses a read only 

attenq)ted to move an always-visible window into a new ^ memory (ROM) and/cr ofeer static storage device 106 

position feat would cause the window to overlap wife coupled to bus 101 for storing static information and instnic- 

anofeer always-^sible window; dons for processor 102. Data storage device 107 is coupled 

FIG. 7 illustrates fee screen FIG. 2 after fee operation to bus 101 for storing information and instructions, 

shown in FIG. 6 showing how the window being moved is Ftmhermore, a data storage device 107 such as a magnetic 

not allowed to overlap wife anofeer always-visible window; 65 disk or optical disk and its corresponding disk drive can be 

HG. 8 illustrates fee screen of FIG. 2 when a user coupled to computer system 100. Computer system 100 can 
attends to resize an always-visible window in such a way also be coupled via bus 101 to a display device 121, sudi as 
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a cathode ray tabe (CRT), for displaying infonnation to a for tiie resource. For example, the abstraction for a window 

con]|>uter user. An alphanumedc input device 122, including indndes information about the size, position, border width 

alphanumeric and other keys, is typically coupled to bus 101 and stacking order of the window (the "configuration" of Ihe 

for communicating infoimation and command selections to window), 

processor 102. Another type of user input device is cursor 5 Because the abstractions are miaintained with the server, 

control 123, such as a mouse, a trackball, or cursor direction only one copy of the abstractions is required, regardless of 

keys for communicating direction information and com- i^qw many clients use the resources. Since clients do not 

mand selections to jsrocessor 102 and for controlling cursor have access to the abstractions themselves, the clients must 

movement on display 121. This input device typically has transmit requests to the server application to manipulate or 

twodegreesoffreedomintwoaxes, a first axis (e.g., x) and lo retrieve information about the resources. When a client 

a second axis (e.g., y), which allows the device to specify transmits a request that relates to a resource, the client 

positions in a plane. identifies the resource with an integer ID that corresponds to 

Alternatively, other input devices such as a stylus or pen the resource. Since only an integer ID is sent rather than a 

can be used to int^act with the display. A displayed object conq)lex abstraction, the traffic between the clients and the 

on a contq)Uter screen can be selected by using a stylus or pen server is reduced, 
to touch the displayed object The computer detects the 

selection by implementing a touch sensitive screen. The Window Manager 

Similarly, a light pen and a light sensitive screen can be used one special purpose client application is ref eixed to as the 

for selecting a displayed object. Such devices may thus window manager. The window manager is a dient appllca- 

detect selection position and the selection as a single opera- 20 ^j^^^ j^^^ certain special responsibiHties. Specifically, the 

tion instead of the **point and click," as in a system incor- window manager mediates competing demands for the 

porating a mouse or tracld>all. Stylus and pen based input physical resources of a display, including saeen space and 

devices as weU as touch and light sensitive screens are well the colormap. Typically, the window manager causes the 

known in the art. Such a system may also lack a keyboard sejyer application to generate a user interface that allows a 

such as 122 wherein aU interface is provided via the stylus 25 ^^^^ windows about on the screen, resize windows, 

as a writing instrument (like a pen) and the written text is and start new appHcations. 

interpreted using optical character recognition (OCR) tech- ^ ^^^^^ appUcation requests a change in the con- 

figuration of a window, the client application sends a request 

The present invention is related to the use of computer to the server. However, since the request relates to a respon- 
system 100 as a platform to execute appHcation programs ^^^^ controlled by die window manager, the sarver appU- 
that support the display of multiple always-visible windows. cation docs not service flie request Instead, die server 
The preferred embodiment of the present invention has been appUcation cancels the request and transmits an event mes- 
designed for use with the X-Windows system. The gage that contains tiie arguments of tiie reconfiguration 
X-Windows system was originally developed by MTT's request to flie window managcar to infoim the window 
Ptoject Athena and Digital Equipment Corporation, and is jnanagcr of the reconfiguration request The vwndow man- 
now managed by the X ConsOTtium. The X-Wmdows sys- ager flien determines whether the modificktions specified in 
temis describedindetailinX/i&/?c/wnceAfawiflt volumes the reconfiguration request should be performed, modified, 
1 and 2, O'Reilly & Associates, Inc. (1995). oj- denied. If the window manager determines that the 

In the X-Windows system, functionality is distributed ^ modifications should be performed, then the window man- 
between dient applications and a server application. The ager sends the reconfiguration request to the server. Because 
client and server applications may be executing on a single the reconfiguration request is from the window manager, the 
processor on a single wcarkstation, or may be distributed server services the reconfiguration request. If the window 
among multiple processors on multiple workstations that are manager determines that the reconfiguration request should 
connected over a network. be modified, then the window manager sends a modified 

. reconfiguration request to the server application. The pro- 

Chent Apphcations ^^^g sending reconfiguration requests to the window 

The client applications transmit requests to the server manager for approval before the server acts upon the recon- 

application. The requests may include requests for tiie figuration requests is referred to as substructure redirection, 

performance of a specified operation, or requests for hafor- 59 

mation. The server application responds to the requests by Wmdow Attnbutes 

performing the specified operation, or by sending a reply to As mentioned above, the server application miaintains 

the client application that includes the requested informa- con^lex data structures that contain information about 

tion. resources. In addition to a wmdow's configuration, the data 

„ . . 55 structure for a window includes information about how the 

server Applications window is to look and act This information is referred to as 

In addition to servicing requests, the searver application window attributes. One attribute of a window is referred to 

sends event messages and errcsr messages to dient applica- as Substructure Redhrect Override. The Substructure Redi- 

tions. Event messages inform the dient applications that rect Override attribute determines whether a window would 

some event has occurred. lypicaUy, the dient applications 60 be allowed to be mapped on the screen without intervention 

are designed to perform some action upon the receipt cf by the window manager, 
certain event messages. Error messages notify the dient that 

a previous request fern the dient was invalid. Always Visible Wndows 

The server plication maintains conq>lex abstractions of The present invention allows a client aj^lication to create 

resources. Such resources may include, for exanaple, 65 and display windows that are always visible. Multiple 

windows, pbunaps, colorm^s, cursors, fonts, and graphics always-visible windows may exist simultaneously without 

contexts. The abstractions for a resource contain attributes defeating the always-visible nature of the windows. The 
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always- visible windows may also exist simultaneously with would be redrawn to cover the portions of the other windows 
any number of normal windows. In this context; normal with which it overlaps. However, according to the present 

windows refers to aU windows that are not always-visible invention, a normal window such as window 210 is never 
windows. The general bdiavior of always-visible windows moved to a higher position in the stack order than any 

shall now be described wilh rdFerence to FIGS. 2-8. 5 currently existing always-visible window. Consequently, the 

selection of window 210 would not cause window 210 to 

Window Behavior move above windows 204 and 206 in the stack order. 

J- » »i. r A u J. . • Therefore, after the selection of the window 210, screen 202 

According to the pt«fetred embodiment of the invention, ^^^j^ iUustrated in FIG. 2, with portions of 

window behavior is governed by two sunple rules. FiKt, ^^^^^ and 206 covering portions of window 210. 

normal windows neva^en positions in the «> ,,i^ti„„ „f ^ tl^^t ^j^^ 

stadc or<ter ttan always-visible windows. As a Kssnlt. no ^^^^ always-visible windows does not cause the 

portion of an always-visiMe wmdow wffi everbe co.^^ ^-^^^^ t„ hi^^ i„ ^ „^ ^ 

obscured by a n«jn«a window. Second, always-vi^le win- always-visible windows, 

dows are prevented from ovcri!q?pu^ wrth each often As a ^^^^ ^ ^ ^^^^^^^ ^ ^^^^^^ ^ 

result, no portion of ^ alway^vmMe wmdow wffl ever be « 8 - 

covered or obscured by any other always-Visible Window. ^ ^-^o i^ ^ '>ao * . L i.- -u . 

^ , ,« "I , ^ "1 ^ o . ^^11 of 208 would cause window 208 to move to the highest 

These rules are lUustratcdin FIGS. 2-9. Refemng to HG. position in the stack order. However, based on the window 

2, It Illustrates a display device 200 that is simultaneously behavior rules implemented in the t^esent invention, win- 

displaymg two always-visible windows 204 and 206 and dow 208 is placed in the highest position in the stack order 

two normal windows 208 and 210 on a screen 202. A visual ^^^^^^ windows, but remains lower in the 

mdicator 212 is also shown on screen 202. Visual indicator g^ack order relative to all existing always-visible windows. 

212 is used by a user to indicate a location on screen 202. Consequently, after window 208 has been resized as illus- 

Visualindicator212moves across screen202inresponse to ^^^^^ ^ 4 ^^^^ ^02 would appear as illustrated in 

user nmmpulation of a user input dew^^ 25 FIG 5. SpedficaUy, window 208 would be placed higher in 

the stack order than window 210, but still lower than 

In the state of screen 202 illustrated in FIG. 2, window windows 204 and 206. Thus wmdows 204 and 206 would 

210 overlaps with and covers a portion of window 208. still cover window 208, but window 208 would now cover 

Windows 204 and 206 overlap with and cover portions of window 210, HG. 6 illustrates an atten^t to cause two 

window 210. No portions of windows 204 or 206 are 3^ always-visible windows to overlap. Specifically, FIG. 6 

covered by any other windows. illustrates the situation in which a user has attempted to 

As mentioned above, when two windows overly, the move window 206 into a new position on screen 202 that 

determination of which window will cover the other hinges would cause window 206 to overlap with window 204. As 

on the current position of the windows in the stack order. mendoncd above, fee present invention prevents two 

The window that is higher in the stack ordor is displayed in 35 always-visible windows from overlapping. Consequently, 

the overlapping region, thereby covering the windows that rather than display window 206 in flie position indicated by 

are lower in the stack order. Consequently, fee state of sacen fee user, fee always-visible window that is being manipu- 

202 in FIG. 2 indicates feat window 210 is currently higher lated by the user is moved back to fee closest non- 

in fee stack order fean window 208, and feat windows 204 overlapping previous position, The effect of feis behavior is 

and 206 are cuirently higher in fee stack order than window 40 that two always-visible windows appear to hnmp into each 

210. ofeer instead of overlapping. FIG. 7 illustrates the position 

Numerous window manipulation operations may cause of window 206 after fee operation performed in FIG. 6. 
two windows to overlap. Fca: exan^le, a window may be Specifically, window 206 is now adjacent to but does not 
moved from one location on screen 202 to anofeer location overlap wife window 204. It should be noted feat fee 
on screen 202. In addition, a window may be resized to 45 position of an always-visible window in fee stack order is 
cover a larger portion of screen 202. Ofeer window mLanipu- irrelevant wife respect to ofeer always-visible windows 
lation operations change fee stack order. When fee stack because always- visible windows are never allowed to over- 
order is changed in such a way as to place a window that is l^p. 

covered by anofeer window higher in fee stack order than FIG. 8 illustrates an attempt to resize an always-visible 
fee window feat is covering it, fee window is redrawn to 50 window in such a way as to cause fee always- visible window 
cover the ofeer window. In typical windowing systems, a to overlap wife anofeer always-visible window. Specifically, 
window must be selected before it can be manipulated. The window 206 has been resized to overlap window 204. As in 
selection process may involve, for example, operating a user fee previous example, fee present invention responds by 
input device to position visual indicator 212 over a visual limiting the resize operation of window 206 to the closest 
portion of a window, and performing a predefined user input 55 non-overlapping previous position. This could result, for 
operation (such as clicking a mouse button). In typical exairq^le, in fee window positions illustrated in FIG. 9. 
windowing systems, fee selection of a window automati- It should be noted that moving fee always- visible window 
cally places fee window at fee top of fee stack order, operated on back to fee closest non-overlapping previous 
However, as shall now be described wife reference to FIGS. position is simply one of many possible techniques for 
3 through 8, fee present invention modifies this standard 50 inq)lementing fee present invention. For example, overlaps 
windowing behavior to support always-visible windows. between always-visible windows may also be prevented by 
FIG. 3 illustrates screen 202 after a user has selected returning fee operated-on always-visible window to its ini- 
window 210. The borders of window 210 feat are cuirently tial state after any operations which would ofeerwise cause 
cova:ed are illustrated with dashed lines. As mentioned fee always-visible window to overlap with anofear always- 
above, in a typical windowing system, fee selection of 65 visible window. 

window 210 would cause window 210 to move to fee Alternatively, overlap between always-visible windows 

highest position in fee stack crder. Therefore, window 219 may be prevented by allowing fee movement of one always- 



described above. 

General System Descdption 
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visible window to "push" other always-visible windows display device 1002 to display in the common screen region 

from their current positions. Thus, if a user attempted to the portion of the window of the two or more windows that 

move window 206 in FIG. 9 upward, window 204 would has a higher position in the stack order than the other 

move upward along with it. Preferably, this upward reposi- windows. For example, windows 1008 and 1010 overlap, 

tioning of windows 204 and 206 would be halted as soon as 5 Because window 1010 is higher than window 1008 in the 

window 204 reached the edge of screen 202. The "push" stack order, window 1010 is displayed in the overlap region, 

technique of preventing overlap between always-visible xhe input reception unit 1020 is connected to the input 

windows requires window management routines that are device 1004 and receives input from a user. The input may 

m<M-e complex than are required by other ov^lap-prevention be, for example, input that designates changes in the 

techniques. TTiis con^lexity is due to the fact that the lo attributes stored in mcmoiy 1014. Window display unit 1018 

movement of one always-visible window may require the indudes a plurality of units for processing the attribute 

movement of any number of other always-visible windows changes. Specifically, window display unit 1018 indudes an 

(e.g. window A pushes window B which pushes windows C overlap processing unit 1028, an overlap coirection unit 

and D. etc.). The present invention is not limited to any 1030, a stack order processing unit 1032 and a stack order 

gjecific manner of in:q)lranenting the window bdiavior rules 15 correction unit 1034. 

The overlap processing unit 1028 determines whether the 
changes cause two or more always-visible windows to 
overlap. The overlap correction unit 1030 alters the plurality 
FIG. 10 illustrates a functional block diagram of a com- of attributes 1022, 1024 and 1026 as needed to prevent 
puter system 1000 for implementing the window behavior overlap between always-visible windows. The stack order 
described above. The computer system 1000 has a display processing unit 1032 determines whether the changes cause 
device 1002, an input device 1004, and a window manage- an always-visible window to have a lower position in the 
ment system 1006. The window management system 1006 stack-order than any window that is not an always-visible 
manages the windows displayed on display device 1002. In window. The stack order correction unit 1034 alters the 
the illustrated example, three windows 1008, 1010 and 1012 ^ attributes 1022, 1024 and 1026 as needed to prevent the 
are currently displayed on display device 1002. always-visible window from having a lower position in the 

Thewindowmanagementsysteml006generanyincludes stack-order than any window that is not an always-visible 
a memory 1014, a window display unit 1018 and an input window. 

reception unit 1020. The memory 1014 contains a plurality 3^ As described above, overiap processing unit 1028 and 
of attributes 1022, 1024 and 1026 of (he windows displayed overlap coirection unit 1030 cooperate to ensure that 
on display device 1002. Hie attributes 1022, 1024 and 1026 always-visible windows are never obscured by other always 
that are stored in nacmory 1014 indicate for each window a visible windows. Similarly, stack ordar processing unit 1032 
stadc ordw position, a screen position, a size (induding and stack order coirection unit 1034 cooperate to ensure that 
vertical and hwizontal dimensions) and whether the window 35 always-visible windows are never obscured by any window 
is an always-visible window. that is not an always-visible window. Consequently, always 

An *'always-visible" attribute is not one of the normal visible windows displayed on display device 1002 are never 
window attributes defined and supported in the X-Windows obscured by any other windows, 
specification. Consequently, a system which includes a ri«*ujTi 
separately-defined always-visible attribute would constitute 40 Client-based Implementation 

an extension to the X-Windows system, rendering the sys- As explained above, in the X- Windows system, window 
tem incompatible with other systems. Therefore, the always- manager clients typically control the rules that govern user 
visible attribute does not exist as a separately-defined win- interface activity. All requests to affect the behavior or 
dow attribute in the preferred embodiment Rather, attributes appearance of windows are first passed by the window 
and/or properties thatpre-exist in the X-Windows system arc 45 manager through substructure redirection. Current window 
used to store the value of the always-visible attribute. managers do not support the behavior of always-visible 
According to one embodiment, the "override-redirect" windows described above. Consequently, one embodiment 
attribute, which is defined in the X-Windows specification, of the present invention overrides substructure redirection 
is used as an "always-visible" flag. In an altemative for always-visible windows by setting the Substructure 
embodiment, a list is attadied as a propcxty of a root window 50 Override Redirect attribute of always- visible windows, 
and used to identify always-visible windows. These embodi- Because the Substructure Override Redirect attribute is only 
ments shaU be described in greater detail below. set for always-visible windows, the value of the Substruo- 

EEaiOillusliatcs&ecn±>odimentinwhiciithcavciridc- Override Redirect attribute also serves as a flag to 

rcdkcct attribute is used as an always-visible window flag. distinguish always-visible windows from noimal windows. 
In the iUustrated example, attributes 1022 include data 55 Once a client application that implements the present 
indicating that window 1008 is not an always-visible invention has identified the always-visible windows, as 
window, and that window 1008 is third in the stack order. diescribed above, the client maintains its own list of always- 
Attributes 1024 include data indicating that window 1010 is visible windows. The list contains infoimation about each 
not an always-visible window, and that window 1010 is always-visible window, induding the window ED, location 
second in the stack order. Attributes 1026 include data 60 and dimensions of the window. 

indicating that window 1012 is an always-visible window The client then tracks all graphics events which may affect 
and that window 1012 is first in the stack order. The window the visibility of the always- visible windows. If any graphic 
display unit 1018 causes display device 1002 to display the event would cause any portion of an always-visible window 
windows 1008, 1010 and 1012 based on the attributes 1022, to become obscured, the client modifies the event to enforce 
1024 and 1026 stored in memory 1014. If portions of two or 65 the window behavior rules described above. Specifically, the 
more windows on display device 1002 share a common client detects when a normal window is placed at the top of 
screen region, then the window display unit 1018 causes the stacking order and immediately transmits requests that 
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cause all always-visible windows to be placed higher in the At step 1204, the X Client creates a window. At step 1208, 
stack order than the normal window. The client also detects the X Client designates that the window created in step 1204 
when an always-visible window is moved or resized. The is an always-visible window. As explained above, the win- 
new position and size is conq>ared with all other always- may be designated as an always-visible window 
visible windows. If the window movement or size change 5 according to one of any number of techniques. One tech- 
causes the always-visible window to overlap with one or ni que is illustrated in FIG. 122?, According to this technique, 
more other always-visible windows, then the client transits the Ov^de-Redirect Attribute of the window is set at step 
a request to reposition or resize the always-visible window 1216. 

to the closest non-overlapping position. An alternative technique for designating a window as an 

_ - 1. J T 1 . 10 always-visible window is illustrated in FIG. 12c. According 

Window Manager-based Implementation ^j^^ technique illustrated in FIG. 12c, the root window has 

Rather than implement the present invention in a client a property which is defined as an always-visible windows 

application that circumvents the window manager to imple- list. At step 1218, the always-visible windows list property 

ment always-visible windows, the present invention may be is retrieved. At step 1220 a window ID that corresponds to 

implemented in a window manager. Similar to the cHent 15 the newly created window is added as an entry into the 

implementation, the window manager would maintain a list always-visible windows list. At step 1222, the property of 

of the always-visible windows and counteract events that the root window that corresponds to the always-visible 

would cause any portion of an always-visible window to windows list is updated to reflect the revised always- visible 

become covered or obscured. However, to prevent other windows list. 

clients from performing operations tiiat would obscure a 20 Refemngagaintona 12a, the window is mapped at step 

portion of an always-visible window, a flag other than the 1210. Mapping the window causes the window to be dis- 

Substwcture Override Redirect would be used to identify played on the ^propriate display. At step 1212, the client 

always-visible windows. For example, a list of always- perforins all dientprocessing. This stq) generally represents 

visible windows (an "always-visible window lisT) may be the operation of the client. The stqps actually performed 

defined and created as a property attached to the root 25 during this step will vary based on the X Client and 

window. interaction between the X Client, the user, and other appU- 

Refening to FIG. 11, it illustrates an embodiment in cations. During the X Client operation, various events may 

which an always- visible window list 1102 is used to identify occur which affect the always-visible window. For example, 

always- visible windows. The always-visible list 1102 other windows (both always-visible windows and non- 

includes a window identifier for each existing always-visible 30 always-visible windows) may be created, mapped, moved, 

window. Specifically, always-visible window list 1102 resized and destroyed. When the X Client is through with all 

includes an entry 1104 that corresponds to window 1012, processing, the X Client destroys the window at step 1214 

wMch is tiie only always-viable window on display device ^ Operation 

1002. During a resize window, select window or move ^ ^ tyiuuvw jYiaugci Kjy^aiiuu 

window operation, window display unit 1018 accesses the 35 According to one embodiment of the invention, a bump 

always-visible windows list 1102 to determine whether whidow manager is used to create the window behavior 

particular windows are always-visible windows. Using an described above. A bump window manager may be a com- 

always-visible window Hst to identify always-visible win- ponent of an X Windows manager or may coexist-exist 

dows allows joint control of always-visible windows by separately from an X Windows manager. The purpose of tixe 

window display unit 1018 and a normal X system window 40 bump window manager is to identify and monitca: windows 

manager. which have been designated as always-visible windows. If 

the stacking order is changed, the bump window manager 

X Client AppHcation Operation ensures that aU always-visible windows are stacked above 

Referring to FIG. 12a; it is a flowchart illustrating the all other windows. If an always-visible window is moved or 

operation of an X CHent application according to an embodi- 45 resized, then the bump window manager win check for 

ment of the invention. The X Client is designed to designate overlapping conditions with other always-visible windows, 

one or more windows as being "always visible". As dis- If such an overlap condition occurs, then the bump window 

cussed above, there are various possible methods for des- manager will issue a move or resize operation to place the 

ignating a window as an always-visible window. Reference always-visible window on which the operation was per- 

to FIGS. 12^ and 12c shall be made to describe two such 50 formed at a previous position or size which does not overlap 

methods. According to the first technique, all windows any other always-visible window. The operation of the bump 

which have the "override redirect" attribute set are "always- window manager will be described in greater detail with 

visible" windows. According to the second technique, a list respect to FIGS. 13a, 13b, 13c, 13d and 13e, 

of window IDS for all always-visible windows is maintained FIG. 13a is a flowchart illustrating the operation of a 

in the form of a prc^rty attached to the root window. When 55 bump window manager. At step 1302, the bmnp window 

this technique is used, each application which creates an manager connects to the X-Scrvcr. This establishes a oom- 

always-visible window is responsible for adding the window munication path between the bun^) window manager and a 

ID to the always-visible window list at the time the window specified X-Server. 

is created. A bump window manager, which will be At step 1304, the bunq) whidow manager uutializes an 

described hereafter, removes the ID of an always-visible 60 always-visible window list Referring to FIG. 13fe, it shows 

window from the always-visible window list automatically the steps performed by the bun^) window manager during 

at tfic tune that the always-visible window is destroyed. step 1304 in greater detaiL Specifically, at step 1314, the 

Referring now to FIG. 12^2, an X client connects to an bun^ window manager queries the X-Server for a list of 

X-Server at step 1202. The X Client connects to tiie windows. At step 1316, the bump window manager identi- 

X-Server by making an Xlib call that establishes a commu- 65 fies the always-visible windows in the list of windows and 

nications path between the X Qient and a designated creates a list of the always-visible windows. Preferably, aU 

X-Server. always-visible windows are children of the root window. 
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The method used by tiie bump window manager to manager receives data that identifies the event that has 
identify which of the windows identiiied by the X-Server are occurred. At steps 1344 to 1358, the bump window manager 
always-visible windows depends on the technique used to identifies the type of event which has occmxed 
designate always-visible windows. For example, if the If the event specifies that a particular window is to be 
override-redirect attribute is used as a flag to designate 5 moved, then control passes from step 1344 to step 1360. At 
always-visible windows, then the bump manager inspects step 1360, the bump window manager determines whether 
the override-redirect attribute of all the windows identified the window on which the operation is to be performed is an 
by the X-Server. If a property attached to a root window is always-visible window. If the window is not an always- 
used to list the always- visible windows, then the bump visible window, then the bump window manager need not 
window manager inspects the appropriate property of the lo Perform any further processing in response to the window 
root window. event If the window is an always-visible window, then 

At step 1318, the bamp window niaiiager qaedes the P^^ff '° f ^ ^^Tfi. At stq. 1376, the bump window 
X-ServaforthesizeaDdpositionofanafttealways-visMe manager deernnnes whether tiie move wiU cause the 
wtodcws.Atstepl320,theb«mpwiiid<>winaiuigcxstcmto «l^?_f "Vf^ible window to overlap with another always- 
an always-vlsible window list the. dzes, p^ons, and 15 viable window. If not^ttien the bump window manag^ need 
window status of each of the always-visible windows. P^rfonn any flirtha processing m response to the 
Consequently, the enHy for each always-visible window in ^"^'^ Otherwise, the always-visible wmdow is 
the always-visible window list includes the whidow ID, "'*^f** '° a non-overlap location m step 1384. The non- 
location, size, status (maptoum?)) and stacking onter of the ^cation may be, for examp e, adjacent to but not 
always-visible window 20 overlapping with the other always-visible wmdow. At step 
L , .J . 1390, the entry in the always-visible window list that 

At step 1322, the bwnp window manager processes the corresponds to the window that was moved is updated to 

new list. The operations performed by bump w-indow man- reflect the new location of the window that was moved, 

ager to process the new always-visible window hst are rr *k. tu^t « ,,^^t^ .^^-a^^. .•- .^ u- 

•11...*. » A rrr/-t 11 t> ^ .• * tt,- it » ti',^ " evcnt q>ecines mat a pararailar window is to be 

f,^ M i^f 13<^- 13yteps 1326, resized, then contol passes ftoiiSq) 134« to step 1362. At 

H38 and 1340 form a loop toat IS repeated unta an overlap step 1362, the bump window inana£r detanrinM whether 

TT "^M'«'l^^7»-;?ible windows IS encountered or a/^wndo^r on whteh the operation! to be performed is an 

until the entries f^: all of the always-visible wmdows have ^j^^ys-visible wfadow. ff fte window is ^an always- 

been processed. The information m an unmapped wmdow > ^ +u - j ^ a. 

cannot be obscured by another window became the infor- window ^en the bump wmdow manager need not 

.r",^"^^ auuui^i y^^^ uc^^auac luc miui perform any further jarocessmg m response to the window 

matron it is not even displayed. Therefore, m the prefeiied fi, o • ^ • i • i>i • ^ 

... . 1 u \a 1 • t_i event. If the wmdow is an always-visible window, then 

embodiment, overlap between an unmapped always-visible i-ito a+ +^ iito *u u • ^ 

^^^A J *if 1 • n.1 • J • • J control passes to step 1378. At step 1378, the bump wmdow 

wmdow and any other always-visible window is isnored* . ^ , t . ^ 

w au a«;r vw*^^ «wa;ro yxoxuK^ wiiiuww x^kji,^ manager detcrmmes whether the resize operation will cause 

When an overlap is encountered between a particular the always-visible window to overlap with another always- 
mapped always-visible window and one or nKxe other 3^ visiblewindow.If not, then the bump window manager need 
mappt^^ways-vijdble windows, th^^ not perform any further processing in response to the 
step 1326 to step 1328. At step 1328, the braap wmdow window event Otherwise, the always-visible window is 
manager determines a new position for the particular resized to a size in which the always-visible window does 
always-visible wmdow which avoids overlap between the overlap with any other always-visible window in step 
always-visible window and aU other mapped always-visible ^ 1386, The new size of the window will be smaller than the 
wmdows. At step 1330, it is determined whether such a ^^^o specified in the resize event. For example, the size may 
position exists. If such a position exists, then the always- be a size which causes the border of the resized window to 
visiblewmdowismovedtotheposittonat step 1336.1f such ^ adjacent to but not overlapping with the border of the 
a position does not exist, then control passes to step 1332. other always-visible window. At step 1392, the entry in the 

At step 1332, the bun^ window manager determines ^5 always-visible window list that corresponds to the window 

whetherthe size limit ofthe always-visible window has been that was resized is updated to reflect the new size of the 

reached. In this context, the size limit is Hie minimum size window that was resized. 

that the always-visible window is allowed to assume. If the jf the event specifies a change in the staddim order, then 
size limit for the always-visible window has been readied, control passes fiom step 1348 to step 1364. At step 1364, the 
then control passes to step 1338 and the next always- visible 50 bvaop window manager determines whether any non- 
window entry in the Ust is processed. If the size limit for the always-visible window is above any always-visible window 
always-visible window has not been reached, then the size in the new stacking order. If no non-always-visible window 
of the always-visible window is reduced. Steps 1326, 1328, is above any always-visible window in ±t new stacking 
1330, 1332 and 1334 form a loop such that the size of an order, then fee bmi^ window manager need not pcrfonn any 
always-visible window will be reduced untQ ciflicr the 55 further processing in response to fee window event. If any 
window can be moved to a non-overlapping position or its non-ahvays-visible window is above any always-visible 
size limit has been reached. window in fee new stacking order, feen control passes to 
Referring again to FIG. 13a, when the always-visible step 1380. At step 1380, fee bump window manager adjusts 
window list has been initialized, fee bun^) window manner fee new stacking order to ensure feat all always-visible 
waits for a window event (step 1306). When a window event eo windows are above aU non-always-visible windows in fee 
occurs, control passes to step 1308 where fee window event stacking order. 

is window event occurs, control passes to step 1308 where if the event specifies feat a particular window is to be 

fee window event is jHrocessed, Stqp 1308 shall be described created, feen control passes fi-om step 1350 to step 1366. At 

in greater detail wife reference to HGS. 13d and 13e. step 1366, fee bun^ window manager determines whefeer 

Referring to FIG. 13d and 13e, feey illustrate a flowchart 65 the newly created window is an always-visible window. Jf 

of fee steps performed by fee bump window manager when fee window is not an always-visible window, feen at step 

a window event takes place. At step 1342 fee bump window 1382 fee bump window manager adjusts fee stacking OTder 
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to ensure that all always-visible windows are above all 
non-always-visible windows in the stacking order. If the 
window is an always-visible window, then control passes to 
step 1388. At step 1388 an entry is added to the always- 
visible window list for tiiie newly created always-visible 
window. 

If the event specifies that a particular window is to be 
destroyed, then control passes from step 1352 to step 1368. 
At step 1368, the bump window manager determines 
whether the window on which the operation is to be per- 
formed is an always-visible window. If the window is not an 
always-visible window, then the bump window manager 
need not perform any further processing in response to the 
window event. If the window is an always-visible window, 
then control passes to step 1396, At step 1396, the always- 
visible window list is updated. Specifically, the entry for the 
always-visible window that is to be destroyed is removed 
from the always-visible window list 

If the event specifies that a particular window is to be 
mapped, then control passes from step 1354 to step 1370. At 
step 1370, the bun^ window manager determines whether 
the window on which the operation is to be performed is an 
always-visible window. If the window is act an always- 
visible window, then the bump window manager need not 
perform any further processing in response to the window 
event. If the window is an always-visible window, then 
control passes to step 1398. At step 1398, the always-visible 
window list is updated. Spedfically, the map/unmap status 
in the entry for the always-visible window is revised to 
reflect that the always-visible window is mapped. Once the 
entry has been revised, the always-visible window list is 
processed at step 1394 to ensure that no mapped always- 
visible windows overlap with any other mapped always- 
visible windows. The steps involved in processing the 
always-visible window list are described above with refer- 
ence to FIG. 13c. 

If the event specifies that a particular window is to be 
unnsapped, then control passes from step 1356 to step 1372. 
At step 1372, the bump window manager determines 
whether the window on which the operation is to be per- 
formed is an always-visible window, the window is not an 
always-visible window, then the bun:p window manager 
need not perform any further processing in response to the 
window event. If the window is an always-visible window, 
then control passes to step 1400. At step 1400, the always- 
visible window list is updated. Specifically, the map/unmap 
status in the entry for the always-visible window is revised 
to reflect that the alw^s-visible window is no longer 
mapped. 

If the event specifies that a property of a particular 
window is to be changed, then control passes firom step 1358 
to step 1374. At step 1374, the bunq) window manager 
determines whether the property to be changed is an always- 
visible window list. If the property to be changed is not an 
always-visible window list, then the buit^ window manager 
need not perform any further processing in response to the 
window event. If the property to be changed is an always- 
visible window list, then control passes to step 1402. At step 
1402 the always-visible window Hst is revised as specified 
in the event. Once the always-visible window list has been 
revised, the always-visible window list is jarocessed at step 
1404 to ensure that no mapped always-visible windows 
overlap with any other mapped always-visible windows. 
The steps involved in processing the always-visible window 
list are described above with reference to HG. 13c. 

Referring again to HG. 13^3; steps 1306, 1308 and 1310 
define a loop in whidi window events are processed as 
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described above until a condition occurs which causes the 
bump window manager to terminate. Upon the occurrence 
of such a condition, control passes from step 1310 to step 
1312, At step 1312, the bump window manager doses the 
5 connection that has been established between the bump 
window manager and the X-Server. 

Transparent Backgrounds 

According to another aspect of the invention, it is dcsir- 
10 able to provide always-visible windows that have a trans- 
parent background. A transparent background allows a user 
to see the screen regions that are covered by the window. 
Only the foreground data within the window is not trans- 
parent By providing always-visible windows with transpar- 
15 ent backgrounds, one can ensure that certain data will 
always be visible and yet minimize the amount of informa- 
tion that is obscured by the always-visible window. 

Possible Applications 

20 The always-viable windows provided by the present 
invention are ideal for use in any computer application that 
requires the constant display of critical informatioiL For 
example, in air traffic control certain data pertaining to 
aircraft must not be covered by any other urformation. 

25 Previously, this constraint did not allow air traffic control 
applications to take advantage of windows-based environ- 
ments. However, by displaying the critical data in always- 
visible windows, air traffic control applications can take 
advantage of the convenience provided by windows-based 

30 graphical environments. Air traffic control is just one 
example of a field which would benefit significantly firom 
the present invention. However, the present invention is not 
limited to any specific field of use. 

While specific embodiments of the present invention have 

35 been described, various modifications and substitutions will 
become apparent by this disclosure. Such modifications and 
substitutions are within the scope of the present invention, 
and are intended to be covered by the following claims. 
What is claimed is: 

^ 1. A method for displaying Information on a display 
device of a computer system, the method comprising the 
steps of: 

simultaneously displaying information in a plurality of 
windows on said display device, wherein said plurality 
of windows includes a first always-visible window; 
maintaining a plurality of configurations that correspond 
to said plurality of windows, wherein the configuration 
for each of said plurality of windows reflects a location 
and dimensions of the window on the display device; 
detecting an event that causes a portion of a second 
window of said plurality of windows to occupy a 
conomion region on said display device with a portion of 
said first always-visible window; 
55 determining whether said second window is an always- 
visible window; 
if said second window is an always- visible window, then 
altering either the configuration of the first always- 
visible window or the configuration of the second 
60 window, or the configurations of both said first 

always-visible window and second window, so that 
no portion of said second window occupies a com- 
mon region on said display device with any portion 
of said first always-visible window. 
65 2. The method of claim 1 fimher comprising the step of: 
if said second window is not an always-visible window, 
then 
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displaying said portion of said first window in said 
common region. 

3. The method of claim 1 wherein: 

said first window has a transparent background; 
said step of displaying said portion of said first window in 
said common region includes the steps of 
displaying in said common region information from 

said first window, and 
displaying in said common region information £rom a 
non-transparent window that overl^s witti said first 
window. 

4. The method of claim 1 further comprising the steps of: 
maintaining a stack order for said plurality of windows, 

wherein each window of said plurality of windows has 
a position in said stack order;, 

receiving user input to cause said second window to have 
a higher position in said stack order than said first 
always-visible window; 

if said second window is not an always-visible window, 
then altering said stack order to cause said first always- 
visible window to have a higher position in said stack 
order than said second window. 

5. The method of claim 1 wherein each of said plurality 
of windows includes an ovcrridc-redirect attribute, wherein 
said step of determining whether said second window is an 
alw^s visible window includes the step of detetmining 
whether the override-redirect attribute is set 

6. The method of claim 1 wherein: 

said plurality of windows include a root window; 
said root window has a property that is defined as an 

always-visible window list; 
said always-visible window list includes an enby for each 

of said plurality of windows that is an always-visible 

window; and 

said step of determining whether said second window is 
an always visible window includes inspecting said 
always-visible window list to determine whether said 
always-visible window list includes an entry for said 
second window. 

7. The method of claim 6 further comprising the steps of: 
detecting an event that destroys said first always-visible 

window; and 

removing an entry that corresponds to said first always- 
visible window from said always-visible window list 

8. The method of clairaL 6 further comprising the steps of: 
detecting an event that creates a new always-visible 

window; 

the method further comprising adding an entry that cor- ^ 
responds to said new always-visible window to said 
always-visible window list; 

determining whether said new always-visible window 
overlaps with any other always-visible window; and 

if said new always-visible window overlaps with any 55 
other always-visible window, then altering either the 
configuration of the new always-visible window or the 
configuration of the other always-visible window, or 
the configurations of both said new always-visible 
VTindow and the other always-visible window, so that 
no portion of said new always-visible window overlaps 
said otficr always-visible window, 

9. A window management system fa: use on a computer 
system that has a display device, the window management 
system coinprising: 

a memory that contains a plurality of attributes of a 
plurality of windows, wherein said plurality of 
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attributes indicate whether each window of said plu- 
rality of windows is an always-visible window, wherein 
said plurality of attributes indicate a stack order posi- 
tion for each window of said plurality of windows; 
an input reception unit for receiving input from a user, 
said input designatmg changes in said plurality of 
attributes; 

a window display unit for displaying said plurality of 
windows on said display device based on said plurality 
of attributes; 

wherein if portions of two or more windows of said 
plurality of wmdows share a connnon screen region, 
then the window display unit displays in said common 
screen region flie portion of the window of said two or 
more windows that has a higher position in said stack 
order than the other of said two or more windows; 
wherein the window display unit includes 
an overlap processing unit for determining whether said 
dianges cause two or more always-visible windows 
to ov^lap; and 
an overlap correction unit for altering said plurality of 
attributes to prevent overlap between said two or 
more always-visible windows; 
a stack order processing unit for determining whether 
said changes cause an always-visible window to 
have a lower position in said stack-order than any 
window tiiat is not an always-visible window; and 
a stack OTder correction unit for altering said plurality 
of attributes to prevent said always-visible window 
from having a lower position in said stack-order than 
any window that is not an always- visible window. 

10. The window management system of claim 9 further 
including a mechanism for maintaining a record of whidi of 
said plurality of windows are always-visible windows. 

11. The window managenaent system of claim 10 wherein: 
said plurality of attributes inchidc an override redirect 

attribute; and 

said medhanism maintains said record by setting the 
override redirect attribute of each of said plurality of 
windows to reflect whether said window is an always- 
visible window, 

12. The window management system of claim 10 
wherein: 

said plurality of windows include a root window; 

said root window has a property defined as an always- 
visible window list; 

said mechanism maintains said record by stcalng an entty 
into said always-visible window list for each of said 
plurality of windows that is an always-visible window. 

13. A method for displaying important information in a 
window-based interface on a screen of a display device in a 
computer system, the method conqjrising the steps of: 

displaying a plurality of windows on said screen, said 
pluralify of windows including one or more always- 
visible windows; 

displaying said important information in an always- 
visible window of said plurality of windows; 

receiving user input that would cause a selected window 
of said plurality of windows to cover a portion of said • 
always-visible window; 

if said selected window is an always- visible window, then 
preventing said selected window from overlapping 
with said always-visible window; and 

if said selected window is not an always-visible window, 
then displaying on said saeen said always-visible 
window on top of said selected window. 
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14. The method of daim 13 further comprising the step of 16. The method of daim 13 further con^rismg the step of 
detecting when said user input would cause said sdected designating said always-visible window as an always- visible 
window to overlap any of said one or more always-visible window by performing the step of setting an override- 
windows by performing the steps of: redirect attribute of said always-visible window. 

storing orientation data that indicates a region of the 5 17. xhe method of daim 13 furtbercoinprising the step of 

screen that corresponds to each of said one or more designating said always- visible window as an always-visible 

always-visible windows; window by perfonning the steps of: 

detecting \v1ien an attenipt is made to display said selected - t., . ^ * ^ 

window in a new re^on on said screen; and ^r^^^g ^ always-visiDle window hst; and 

comparing the new region with the orientation data to adding to said always-visible window Ust an entry that 

determine if the new region intersects with the region corresponds to said always-visible window, 

of the screen that corresponds to any of said one or 18, The method of claim 17 wherein said step of creating 

more always-visible windows. an always-visible window list includes establishing said 

15. The method of daim 13 further comprising the step of: always-visible window list as a property of a root window, 
maintaining a stack order for said plurality of windows; method of claim 17 wherem said entry includes 
displaying said plurality of windows on said screen a window identifier, a location indicator, a size indicator and 

responsive to said stack order, wherein if any windows ^ stacking order indicator for said always visible window, 

of said pluraUty of windows overlap, then the window ^^^^ 

with a higher position in said stack order IS displayed 20 ^^v- . , 

above the^indows with which it overlaps; .^^^a that indicates whether said always visible 

detecting when said user input would cause any window wmdow is mapped ^ ^ 

that is not an always-visible window to assume a highw ^1. A con^uter-readable medium that has stored thereon 

position in a stack cnrder than any of said one or more sequences of mstructions which, when executed by a 

always- visible windows; and 25 processor, cause the processor to perform the method recited 

causing said any window to assume a position in said in claim 13. 
stack order that is lower than all of said one or more 

always-visible windows. ***** 
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SELECTIVELY CONTROLLABLE HEADS-UP As explained in greater detail below, prior attempts in the 

DISPLAY SYSTEM medical field to fulfill that need have been unsatisfactory. 

For example, video signal sources have been adapted to 
scan views or images for different types of medical uses and 

FIELD OF THE INVENTION 5 applications. U.S. Pat. No. 4,737,972 to Schoolman 

^ . . 1 . . 1 . . J. 1 . J ("Schoolman T') discloses a head-mounted device that pro- 

Hie invenuons relate to electronic display systems and V^^^ stereoscopic x-ray images. Furthermore, U.S. Pat. No. 

control systems therefor. More particularly the inventions 4,651,201 to Schoolman ("Schoolman IF') discloses an 

relate to selectively operable heads-up display systems for endoscope that provides stereoscopic images of the patient 

presenting information and/or image(s) to the user. In its on a display. Both Schoolman I and Schoolman II allows for 

preferred form, the heads-up display is configured for use by selective transmission of other video data to the display, 

medical technicians or personnel, such as surgeons perform- However, Schoolman I and Schoolman II do not use a "see 

ing an operation. through" display that allows the surgeon to monitor both the 

BACKGROUND OF THE INVENTION environment around him and the video image. If the surgeon 

^5 wishes to monitor or view the real- world environment, as 

A heads-up display is generally defined as an electroni- opposed to the displayed graphics, the head-mounted dis- 

cally generated display containing information or data that is play must be removed. 

superimposed on an observer's normal field of view. As Efforts have also been made to use head-mounted displays 

explained in greater detail below, heads-up display ("HUD") augmented reality simulations for medical applications 

systems have been used in various applications. One such wherein a desired image or three-dimensional model is 

application is for use by pilots of aircraft. In the typical " superimposed on the real scene of a patient. For example, it 

aircraft HUD system, a semi-transparent display screen is ^^s reported that a research effort in the Department of 

located generally in front of the eyes of the pilot (i.e. a screen Computer Science at the University of North CaroUna has 

mounted on the pilot's head or helmet, or in the view of the attempted to develop a see-through head-mounted display 

aircraft windshield). Such a system enables a pilot to con- that superimposed a computer-generated three-dimensional 

centrate on the difficult tasks associated with flying the i^^ge of the internal features of a subject over the real-life 

aircraft, without diverting his attention to scan or examine a view of the subject. Information describing those research 

wide array of instruments. efforts may be found on the World Wide Web in a document 

It is also well known that medical technicians or maintained by Jannick Rolland at the site on the World Wide 

personnel, such as surgeons, must keep track of many 3^ Web Pages of the NSF/ARPA Science and Technology 

different types of information during an operation. For Center for Computer Graphics and Scientific Visualization 

example, a surgeon must carefully view or monitor the at the University of North Carolina, Chapel Hill (http:// 

physical surgery while simultaneously monitoring a www.cs.unc.edu/~rolland, cited February, 1996, copies of 

patient's condition (e.g., blood pressure, heart rate, pulse, which are included in the information disclosure statement 

etc.). In addition, depending on the procedure, the surgeon 35 that has been filed concurrently with this application). That 

must also monitor the status and settings of surgical equip- Wodd Wide Web site in turn referenced the following 

ment and tools. Although the additional information is publication: A, R. Kancheral, et al., "A Novel Virtual Reality 

necessary and important, monitoring such information often Tool for Teaching 3D Anatomy," Prac. CVR Med '95 (1995). 

diverts the surgeon from the immediate task at hand. Other research efforts al the University of North Carolina 

Some surgical operations require that the surgeon divert 40 attempted to use a video see-through head-mounted display 
his eyes to view a video monitor, for example, when and a high-performance computer graphics engine to super- 
performing highly complex laser or internal surgery con- impose ultrasound images over the real view of the subject, 
ducted through scopes. See U.S. Pat. No. 5,222,477, which thereby allowing a user to "see within" the subject. A set of 
discloses an endoscope or borescope stereo viewing system. trackers captured the motion of the body part with respect to 
In addition, the surgeon may from time to time need to refer 45 the field of view of the user, and a computer updated the 
to data, such as defined by a patient's recorded or written position of the body part in real time. The computer 
history, or to previously taken x-rays or other computer attempted to correlate the "tracked" position of the body 
generated images (e.g., CAT, NMR, 3D, etc.). For example, with the three-dimensional model and to display the model 
U.S. Pat. No. 5,452,416 discloses an automated system and on the heads-up display in a manner that gave the appear- 
a method for organizing, presenting, and manipulating medi- 50 ance of "x-ray vision." 

cal images for viewing by physicians. See also U.S. Pat. In the above -described University of North Carolina 

Nos. 5,251,127 and 5,305,203, which disclose a computer- research efforts, the focus was primarily to help teach 

aided surgery apparatus that positions surgical tools during students by superimposing a single computer-generated 

surgery or examinations. In each of the above-described image over a moving, real-life, image of a subject. However, 

systems, in order to view the displayed information, the 55 as explained in the associated literature, the "tracking" 

surgeon must divert his or her eyes to a remote monitor. requirements made the research effort quite complicated. 

Thus, surgeons use many different types of displays and and the results appeared less than satisfactory. Moreover, 

must continually monitor many different sources of infor- such a teaching system is not applicable to the real-world 

mation. However, as more tools and sources of data become environment of a surgeon, where the patient is not moved 

available to surgeons for use during operations, more oppor- 60 (and "tracking" is unnecessary), and where the surgeon 

tunities for distraction arise. It is difiScult for a surgeon to needs or desires other information to be made readily 

focus on his or her conduct during a surgical procedure available for viewing. 

while also continually shifting focus away from the patient Still another research program associated with the Uni- 

to other monitors or indicators. Therefore, a need exists for versity of North Carolina is described in Fuchs, et al., 

conveniently, efficiently and accurately displaying to a sur- 6S "Virtual Space Teleconferencing using a Sea of Cameras," 

geon various types and sources of information, views, and Proceedings of the First International Symposium on Medi- 

images of a patient undergoing a critical medical procedure. cal Robotics and Computer Assisted Surgery (Pittsburgh, 
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Pa., Sep. 22-24, 1994). That article describes research 
efforts that attempted to use a multitude of stationary cam- 
eras to acquire both photometric and depth data. The 
acquired data was purportedly used to constmct a remote 
site in accordance with the head po.sition and orientation of 
a local participant. According to the article, each participant 
wears a head-mounted display to look around a remote 
environment having surface geometries that are continu- 
ously sensed by a multitude of video cameras mounted along 
the waUs and ceiling, from which cameras depth maps are 
extracted through cross- correlation stereo techniques. Views 
acquired from several cameras are then displayed on a 
head-mounted display with an integrated tracking system to 
provide images of the remote environment. The explained 
purpose of the effort was to duplicate, at a remote location, 
a three-dimensional virtual reality environment of a medical 
room. However, the article does not disclose the use of 
see -through displays providing a surgeon with the ability to 
select and display additional forms of data, or to superim- 
pose data over a real-life view of the patient or surgical site. 

Another type of head-mounted display is described in 
Yoshida, et al., "Optical Design and Analysis of a Head- 
Mounted Display with a High-Resolution Insert," Proc. 
SPIE 2537 (1995), That article describes yet another 
research program associated with the University of North 
Carolina in which a small area of a high-resolution image is 
inserted on a large field of a low resolution image displayed 
on a head-mounted screen. The system is described as using 
eye-tracking information to dynamically place the high 
resolution insert at the user's gaze point. The system pur- 
ports to provide the user with both high-resolution imagery 
and a large field of view. In essence, using eye-tracking 
electronics, the "inserted image*' corresponding to the user's 
gaze point is converted from low resolution to high- 
resolution. However, as above, the user can not select 
additional or alternative forms of data or different images to 
be superimposed over the primary image on the head- 
mounted display. 

Thus, few head-mounted displays have been developed 
for the medical industry, and all those described above have 
had limited purpose and utility. On the other hand, and as 
discussed briefly above, a wide variety of head-mounted 
devices are commonly used in military applications. As 
mentioned, aircraft pilots, tank commanders, weapon opera- 
tors and foot soldiers have all used head-mounted displays 
to display various forms of weapon or image information 
along with other data defining the real-world environment of 
the person wearing the display. For examples of such 
systems, see the following U.S. Pat. Nos. 4,028,725; 5,281, 
960; 5,000,544; 5,227,769; 4,994,794; 5,341,242; 4,878, 
046; 3,940,204; 3,923,370; 4,884,137; 4,915,487; and 
4,575,722. Likewise, helmet or head-mounted displays have 
also been used for motorcycle riders. U.S. Pat. No. 4,988, 
976 discloses a motorcycle helmet that displays data or 
information such as speed, time, rpm's, fuel, oil, etc. on the 
transparent visor (i.e. vacuum fluorescent display) of the 
rider. Head-mounted displays that are worn in front of the 
user*s eyes or worn as eye spectacles also exist. For 
example, see the following U.S. Pat. Nos. 5,129,716; 5,151, 
722; 5.003,300; 5,162,828; 5,331,333; 5,281,957; 5,334, 
991; 5,450,596 and 5,392,158. 

The field of virtual reality also has driven advances in the 
use of various types of head-mounted displays. For example, 
see the following U.S. Pat, Nos. 4,636,866; 5,321,416; 
5347^400; 5,348,477; 5,406,415; 5,414,544; 5,416,876; 
5,436,765; 5,479,224; 5,473,365; 0363,279; 5,485,172; 
5,483,307; 5,130,794. See also the publication //ow Virtual 
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Reality Works, by J. Eddings (Ziff-Davis Press, Emeryville, 
CaUf., 1994), and the site maintained by Rolland (referenced 
above) relating to telepresence systems and augmented 
reahty. 

5 Advances have also been made in the area of heads-up 
displays or screens that are not attached to or worn by the 
user. Most commonly, such systems are employed in auto- 
motive or military environments, to provide vehicle 
performance, weapon status, and other data for the driver or 
pilot. For examples of such systems, see the following U.S. 
Pat, Nos. 5,278,696; 4,652,870; 4,711,512; 4,729,634; 
4,799,765; 4,927,234; 4,973,139; 4,988,976; 4,740,780; 
4,787,711; 4,740,780; 4,831,366; 5,005,009; 5,037,182; 
5,231,379; 4,824,228; 4,763,990; 4,669,810; 4,688,879; 

^5 4,818,048; 4,930,847; 4,932,731; 5,198,895; 5,210,624; 
5,214,413; 5^02,964; 4,725,125; 4,188,090; 5,028,119 and 
4,769,633. 

Numerous advances have occurred in the specific forms 
of, and materials used in, heads-up display systems. See, for 
20 example, U.S. Pat. Nos. 4,987,410 and 4,961,625 (use of 
Liquid Crystal Displays (LCDs)); U.S. Pat, Nos. 5,108,479 
and 5,066,525 (laminating glass plates or panels); and U.S. 
Pat. No, 5,457356 (making a flat panel head-mounted 
display). 

25 Also pertinent to this invention is the field of eye-tracking 
to control various computer or imaging functions. Various 
systems unrelated to the medical field have used eye- 
tracking for controlling a field of view. For example, see 
U.S. Pat. No. 4,028,725, which discloses an eye and head 

30 tracking system that controls a beam-splitter and retains the 
high-resolution part of the image in the field of view. The 
eye-tracking is carried out by infrared detection (i.e. see U.S. 
Pat. No. 3,724,932). See also U.S. Pat. Nos. 5,287,437; 
4,439,755; 4,349,815; 4,437,113; 4,028,725 (referenced 

35 earlier) and the article "Optical Design and Analysis of a 
Head-Mounted Display with a High-Resolution Insert," 
referenced above, particularly at footnote 19, which refers to 
the Eye-tracking Systems Handbook, Applied Science 
Laboratories, Waltham, Mass. (1992). 

40 Finally, video recording systems for recording scenery 
and heads up displays have been taught by the prior art. U.S. 
Pat. No. 5,241391 to Dodds ("Dodds") discloses a video 
camera system that records scene conditions and heads-up 
displays. 

45 Notwithstanding the large number of articles and patents 
issued in the area of heads-up or head-mounted displays, 
there has been no such display that is designed for the 
special needs of individuals performing detailed but critical 
tasks on relatively stationary subjects. Such a system would 

50 be extremely use foil to personnel working in the fields of 
medicine, forensics, and micro-electronics. 

Presently, there is a need for a selectively operable, 
head-mounted, see-through viewing display system for pre- 
senting desired information and/or images to a user, while at 

55 the same time allowing the user to view the real-world 
environment in which he or she operates. There is a further 
need to provide a convenient selectable viewing system that 
can be easily controlled by an eye-tracking cursor and 
speech recognition to control different images or displays on 

60 a video monitor or to control a field of view, while keeping 
the user's hands free to conduct precision operations. 

SUMMARY OF THE INVENTION 

Accordingly, it is an object of this invention to provide an 
65 improved heads-up display system. 

It is an other object of the invention to provide a "hands- 
free" heads-up display system that is useful to individuals 
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performing detailed procedures, such as those working in desired data. The computer communicates with and controls 

the fields of medicine, forensics, micro-electronics, biotech, the heads-up display system, which is configured to display 

etc. the desired data in a manner that is aligned in the user's field 

It is another object of the invention to provide an heads-up display includes a user interface 

improved head-mounted display that allows the user to view 5 incorporating "hands-free" menu selection to allow the user 

both the subject and selected data. tocontrolthedisplayofvanoustypesof data. In its preferred 

torm, the hands-tree menu selection is carried out using an 

It is another object of the invention to provide an eye-tracking cursor and a speech recognition computer to 

improved heads-up display that mcludes a user-friendly point to and select specific menus and operations, 

interface to a command control computer. Tj,e above and other objects are also achieved in an 

It is another object of this invention to provide an user-controllable heads-up system for presenting medical 

improved heads-up display that interfaces with a command data to a physician. The system includes a command control 

control computer and includes an eye-tracking cursor to computer for receiving inputs defining medical data and for 

select menus to control computer performance and the controlling the display of that data on a head's-up display 

display of data. screen in the normal field of view of the physician. The 

It is another object of this invention to provide an heads-up display provides the physician with a "user inter- 
improved heads-up display that interfaces with a command face" including menus and associated operations that can be 
control computer and includes a speech recognition circuit selected with an eye-tracking cursor. The system also 
to control computer performance and display of data. includes a microphone and speaker so that a physician can 

It is another object of the invention to provide an 20 communicate with other personnel and computers both 

improved heads-up display that can be positioned between a ^^^^^^y ^^^^^^ from the site. The command computer 

surgeon and a patient in the surgeon's line of sight. mcludes a speech recognition processor to respond to spo- 

. 1 • . r • 4 -J ken commands of the physician. The command computer 

It is another object of the mvention to provide an , . j • ^ . 

J I. J :4- 1 41. * 11 *i. ; • *u also communicates with and receives a wide array or data 

improved heads-up display that allows the user to view the ^ . . i j *i. -.u u 

if. . I.., . "^i • .1 * « XL. nz irom Other computers networked therewith. The physician 

subject while simultaneously monitonng the output firom a 25 j .u i 

. c j-rc c *• ■ 1 J- • • can select the speciiic data to be displayed on the screen. In 

number of different mformation sources, including unaging , i, • • -.u *u * i- 

J. J * * oo-o addition, Ihe physician can, with the eye-tracking cursor, 

devices and remote or networked computer systems. i -, ! ' i- 

control vanous medical imaging devices. 

It is another object of the invention to provide an ^^^^^ ^^^^^ ^^^^^^ ^1^^ ^^1^.^^^^ ^ ^^^1^^^ 

improved heads-up display that allows a medical technician selectively displaying mukiple forms of data on a head- 
to control medical imaging devices to obtam images of ^. accordance with the method, a see- 
select parts of a patient and to display those images on the ^^^^^^ ^^^^^^^^ ^^^^^^ ^^^^^ ^ ^^^^^^^ ^ ^^^^ p.^^^ 
ea s-up isp ay. ^ ^^om by the user. A command computer controls a 

It is another object of the invention to provide an user interface so that command icons or menus are displayed 
improved heads-up display that allows a user to control a 3^ in a super-imposed manner on the see-through, head- 
computer to acquire and display data defining a subject^s mounted display, thereby allowing the user to see both the 
history while simuUaneously viewing the subject. normal field of view and the user interface. The user 

It is another object of the invention to provide an interface is provided with a "point-and-select" type of 
improved method for conducting surgery on a patient while cursor. An eye-tracking system is integrated with the com- 
simultaneously obtaining access to and conveniently dis- 40 mand control computer and the user interface to monitor the 
playing on a heads-up display a variety of types of data user's eye movement and to correspondingly control move- 
relating to the patient. ment of the cursor. The user selects various computer 

It is another object of the invention to provide an operations from menus contained in the user interface by 

improved method of controlling a heads-up display by moving the eye-tracking cursor to selected menus or icons, 

employing a "point-and-click" type user interface and cursor 45 By using the eye-tracking cursor to select various computer 

controlled by tracking movement of the eye. operations, the user can control the command computer to 

It is another object of the invention to provide an selectively disjlay on the see-through HUD screen numer- 

improved method of controlling a heads-up display by ^^^^ ^"^^S^^' ^^^^^ ^^^^ 

employing speech recognition, both alone and in combina- ^ 

tion with an eye-tracking cursor. 50 The preferred embodiments of the inventions are 

. u- . 4^ ,u ' . J described below in the Figures and Detailed Description. 

It is another object 01 the mvention to provide an ^- , */i „ . j -f • - . r.i. • . 

, V 1 , ♦u i 11 4U ♦ Unless specifically noted. It is the mtention of the inventors 

improved heads-up display system that allows the user to . .i_ j Ji i_ • .i_ •/= j 1 • u 

1 , , . , f J J u A that the words and phrases m the specification and claims be 

control tools or instruments in a hands-free manner. . j. j . j • . .1. i? 

given the ordinary and accustomed meaning to those of 

It IS another object of the invention to provide an ordinary skill in the applicable art(s). If the inventors intend 

improved heads-up display system that allows a surgeon to ^ny other meaning, they will specifically state that they are 

comrol surgical tools or other instruments in a hands-free applying a special meaning to a word or phrase, 

manner. Likewise, the use of the words "function" or "means" in 

It is another object of the invention to provide an the Detailed Description is not intended to indicate a desire 

improved heads-up display maintained in an eyepiece of a go to invoke the special provisions of 35 U.S.C. Section 112, H 

scope or instrument and that is controlled with integral 6 to define his invention. To the contrary, if the provisions 

eye-tracking and speech recognition systems. of 35 U.S.C. Section 112, 116 are sought to be invoked to 

The above and other objects are achieved in an improved, define the inventions, the claims will specifically state the 

selectively controllable system for presenting desired data phrases "means for" or "step for" and a function, without 

on a head-mounted (or "heads-up") display. The system 65 also reciting in such phrases any structure, material or act in 

includes a command computer processor for receiving support of the function. Even when the claims recite a 

inputs that represent data and for controlling the display of "means for" or "step for" performing a function, if they also 
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recite any structure, material or acts in support of that means such as used in micro-electronics, biolech, medicine, 

of step, then the intention is not to invoke the provisions of forensics, chemical research, etc. 

35 U.S.C. Section 112 f 6 Moreover, even if the inventors Similarly, the specific arrangement of the icons and menus 
invoke the provisions of 35 U.S.C. Section 1X2, H 6 to define j^at appear on the HUD screen, and the associated opera- 
he inventions, it IS the intention that the inventions not be 5 Uons performed by those icons and menu items, are a matter 
limited only to the Specific Structure, material or acts that are u • c -xi i- rp. ^. . 
described in his preferred embodiments. Rather, if the claims «f choice for the specific apphcation. Thus, the invention is 
specifically invoke the provisions of 35 U.S.C. Section 112, intended to be hmited to the specific arrangement and 
H 6, it is nonetheless the intention to cover and include any moments of the icons and menus shown and described in the 
and all structures, materials or acts that perform the claimed Preferred embodiments. For example, the icons and menu 
function, along with any and all known or later developed ^ se ectively controllable heads-up display used by 
equivalent structures, materials or acts for performing the f ^^"^^^^ ^^^^^^ ^^^^^^ ^^^^^^"t arrangement used 
claimed function. ^ micro-electromcs engineer. 

As a primary example, the preferred embodiment of this ^^^^htr examples exist throughout the disclosure, and it is 

invention is configured for use by a surgeon performing an the intention to exclude from the scope of the invention 

operation on a patient. However, the invention is equally the use of structures, materials or acts that are not expressly 

applicable to any environment in which the user is conduct- identified in the specification, but nonetheless are capable of 

ing precision or detailed procedures with his or her hands on performing a recited function, 

a relatively stationary subject, and where the user would find BRIEF DESCRIPTION OF THE FIGURES 

it advantageous to see data superimposed over the normal 20 

field of view. The potential applications are too numerous to The inventions of this application are better understood in 

mention, but would include forensics, microelectronics, conjunction with the following Figures and Detailed 

biotechnology, chemistry, etc. Thus, even though the pre- Description of their preferred embodiments. The various 

ferred embodiment refers to use by a surgeon, and to the hardware and software elements used to carry out the 

acquisition and display of medical data, its applicability is inventions are illustrated in the attached drawings in the 

much broader, and the claims should be interpreted accord- " form of block diagrams and flow charts. For simplicity and 

ingly. brevity, the Figures and Detailed Description do not address 

Further, the description of the preferred embodiments ^ f^^t^^l features that are well known in the prior art, such 

make reference to standard medical imaging devices that are the Hterature listed in the Background of the Invention, 

used to generate images to be displayed on the heads-up 30 above. However, to assure an adequate disclosure, the 

display. The disclosure specifically references several specification hereby expressly incorporates by reference 

examples of such devices, including video cameras, x-ray ^^^^ ^^ery patent and other pubhcation referenced 

devices, CAT and NMR scanners, etc. However, numerous above in the Background of the Invention, 

other medical imaging systems are well known to exist, and FIGS. lA, IB and IC depict side, top and front views, 

most likely, numerous improved imaging devices will be 35 respectively, of a selectable heads-up display worn by a 

developed in the future. Thus, the present invention does not technician, such as a surgeon. 

depend on the type of imaging device that is implemented. FIG. 2 shows an example for a basic configuration of an 

The inventions described herein are not to be limited to the integrated head mounted display (HMD) and associated 

specific scanning or imaging devices disclosed in the pre- computer system used by a technician, such as a surgeon. 

ferred embodiments, but rather, are intended to be used with 40 FIG. 3 is a block diagram of the primary components of 

any and all applicable medical imaging devices. Likewise, the heads-up surgeon*s display of FIG. 2 

the preferred embodiments depicted in the drawings show a 4 ^ ^^^^ ^^^^^^^ ^^^^^ ^- ^ ^^^^^^^ 

single generic unagmg device mounted on a manipulator embodiment of the heads-up display system. 

arm. Numerous other tool and manipulator configurations, ™^ i- , . . ,1. r .1 . . , 

and multiple imaging devices, can be substituted for the 45 . ^Jf' ^ ^T'^ embodmient for the physical eye- 

single device trackmg system implemented with an infrared laser. 

Further, the specification in some places refers to several ^'^"^ ^" embodiment for the display view and 

computers or controllers that perform various control opera- associated icons and menu items as seen by the surgeon 

tions. The specific form of computer is not important to the ^^^""^ *^ 1^^^^^-"? *«P'^y °" ^ transparent or see-through 

• screen 

invention. In its preferred form, applicant divides several of 50 

the computing, control and analysis operations into several depicts an embodiment for the display view and 

cooperating computers or embedded systems. However, associated icons and menu items as seen by the surgeon 

with appropriate programming well known to those of wearing the heads-up display, with a patient's vital signs 

ordinary skill in the art, the inventions can be implemented selected for display on a transparent or see-through screen, 

using a single, high power computer. Thus, it is not the 55 FIG. 6C depicts an embodiment for the display view and 

intention to limit the inventions to any particular form or any associated icons and menu items as seen by the surgeon 

number of computers, or to any specific computer network wearing the heads-up display, with a view from one of the 

arrangement. cameras selected for display on a transparent or see-through 

Likewise, the detailed description below shows at least screen, 

two embodiments for the display screen. The preferred 60 FIG. 7A depicts an embodiment for the display view and 

embodiment discloses the display screen mounted on the associated icons and menu items as seen by the surgeon 

head of the user. The second embodiment shows the display wearing the heads-up display, v/ith a view from one of the 

screen positioned between the user and the subject, in a cameras selected for display on a portion of the screen that 

manner that is not mounted upon or supported by the head has been made selectively non-transparent, 

of the user. Additional embodiments also exist, and need not 65 FIG. 7B depicts an embodiment for the display view and 

be disclosed. For example, the first embodiment can be associated icons and menu items as seen by the surgeon 

modified for use in the eye-piece of a scope of any form, wearing the heads-up display, with multiple x-ray images 
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from the computer or one of the cameras selected for display while performing an operation on a patient 34. In a preferred 

on a portion of the screen that has been made selectively form, the surgeon 12 can selectively control the display 

non-transparent. screen 14 to operate between opaque or translucent modes. 

FIG. 7C depicts an embodiment for the display view and ^he opaque mode, the screen 14 will display primarily the 

associated icons and menu items as seen by the surgeon 5 data or images generated from a control computer 36 or one 

wearing the heads-up display, with multiple forms of data °' °' ? 'T!, T'?^'^ f ' \ 

J . 1' r.i example, when conducting particularly detailed or internal 

and images displayed in windows on a portion of the screen. ] , i . ^ i-» n u ui 

^ ^ surgery. In the translucent mode, the surgeon 12 will be able 

FIG. 7D depicts another form for the display view and through display screen 14 to the patient 34, while at 

associated icons and menu items as seen by the surgeon the same time seeing the data or images generated by the 

wearing the heads-up display, with multiple forms of data computer 36 or imaging input device(s) 38. 

and images displayed in various windows on a portion of the ^.^pj-^y disclosure, only one imaging input device 

screen, shown in FIG. 2. However, it is expressly noted that 

FIGS. 8A, 8B and 8C depict various views of an alter- multiple numbers, and any and all forms, of image- 
native embodiment implemented with a transparent heads- generating systems can be employed in the proposed system, 
up display that is not worn by the surgeon, but rather, is Thus, CCD, video, x-ray, NMR, CAT, and all other medical 
movably stationed over the patient. imaging systems can be employed. In its most basic form, 
DETAII ED DEvSCRIPTION imaging input device 38 is mounted on a moveable and 

computer controllable manipulator assembly or arm 39, so 

Shown in FIGS. lA, IB and IC are three views of a 20 that it can be controlled by the surgeon 12 through the HUD 

head-mounted, selectable, display system 10. In the pre- system to move to selected parts of the patient 34, and to 

ferred embodiment shown in the Figures, the display system obtain and magnify images of particular parts of the body, 

10 is worn by a surgeon 12 performing an operation, tissue or organs undergoing surgery. For a more detailed 

However, the system is easily programmed for use by any discussion on automatically controlled and moveable 

individual performing detailed procedures in which it is 25 cameras, see U.S. Pat. No. 5,249,045 and the patents cited 

advantageous to see the normal field of view, while also therein, all of which are incorporated herein by reference, 

having access to and seeing in that field ofview a variety of in operation, the surgeon 12 can command the HUD 

forms ofdata relating to the procedure. Thus, it is envisioned system 10, under control of computer 36, to selectively 

that a display in accordance with this invention will have display on screen 14 various forms of data, graphics or 

many applications, but is primarily intended for procedures 3Q images, including magnified or othenvise modified images 

where detailed work is performed on relatively stationary from the imaging input device(s) 38, while at the same time 

objects. Accordingly, while the description below refers looking through the screen 14 to view the patient 34 and the 

repeatedly to the user as a "surgeon", it should be understood normal, real-life environment of the operating room. Thus, 

that the other users are included in the scope of the inven- the surgeon 12 is able to directly view the patient 34, while 

tio"- 35 at the same time, select from many forms of data for display 

For convenience, the phrases "head-mounted display," on the HUD screen 14. For example, if the surgeon 12 is 
"heads-up display" and "HUD" are used interchangeably performing surgery in close proximity to critical organs, the 
throughout this specification. The major components of the surgeon may wish to see both the movements of his or her 
HUD system 10 as worn by the surgeon 12 include a display hands, and a superimposed magnified view from one of the 
screen 14, microphone 16, speaker 17, display driver 18, 40 image input device(s) 38. The surgeon 12 can control the 
camera 29, display mirrors 20 and 22, light 31, eye-tracking video device 38 to move to and focus on the critical area or 
laser 24, eye-tracking detector 26, and eye-tracking optics surgical site, and to obtain and magnify and image for 
28. In its preferred form, each of these components are display on the HUD display 14. In addition, the sturgeon can 
integrated into a single, adjustable head piece that is placed control the computer system 36 to record the generated 
on the surgeon's head 30. Once placed on a surgeon's head 45 images, and then to display on the HUD screen 14 selected 
30, the various components are adjusted for proper opera- parts thereof after being magnified or otherwise computer 
tion. More specifically, the screen 14 is positioned comfort- enhanced. In accordance with the invention, the surgeon 12 
ably in front of and in the normal line of sight of the has the great advantage of being able to simultaneously 
surgeon's eyes 32, and the microphone 16 (placed in front of control the HUD and data acquisition systems, while also 
surgical mask 19) and speaker 17 are positioned so that the 50 looking through the HUD display 14 to watch minute hand 
surgeon 12 can communicate with selected medical assis- movements and other aspects of the local environment, 
tants and computers including an electronic speech recog- As explained in greater detail below, the surgeon 12 may 
nition system, as discussed in greater detail below. The command the computer 36 in several ways. In the preferred 
display driver 18 and display mirrors 20 and 22 are adjusted mode, the HUD system 10 incorporates eye-tracking to 
to superimpose selected images and data with light rays 4 55 control a cursor that is displayed on the HUD screen 14. 
from outside of HUD 10 on the display screen 14 via optical More specifically, as shown graphically in FIGS. 6A--6C, the 
path 6. Likewise, the eye-tracking laser 24, eye-tracking standard display 14 includes menu items or icons that can be 
detector 26, and eye-tracking optics 28 are aligned to detect selected by a cursor 40 that is in turn controlled by an 
and communicate eye movement to an eye-tracking eye-tracking system. For example, when the surgeon 12 
computer, as also discussed below. In more compact ver- 60 focuses his eyes on a selected icon or menu displayed on the 
sions of the head mounted display system discussed above HUD screen 14, the eye-tracking system will correspond- 
and shown in FIGS. lA, IB, IC, and 2, an eyeglass frame ingly cause the cursor 40 to move or "track" over the 
may be used to support the display surface 14, eye-tracking selected icon or menu item. The surgeon can then select the 
laser 24, eye-tracking detector 26, eye-tracking optics 28, or specific icon or menu by voice command or with a foot- 
other components as desired. 65 operated select button (not shown) operating in a manner 

Referring to FIG, 2, the surgeon 12 wears the HUD similar to the well known "mouse" button. Thus, the surgeon 

system 10 to simultaneously view selected data and images operates eye-tracking cursor 40 in a "hands-free" manner to 
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move onto icons and to select various imaging systems, 
computers or other data sources for display on the HUD 
display 14, 

Alternatively, or in combination with the eye-tracking 
cursor 40, the HUD system 10 includes a standard speech 5 
recognition sub-system integrated with the command opera- 
tions. Thus, the surgeon 12 can speak select speech com- 
mands or select words to select specific menus or to initiate 
computer operations to acquire and display select images or 
data. In the combined speech and eye-tracking mode, the ^0 
surgeon can use his or her eye to move the cursor 40 to a 
particular menu or icon, and then speak commands to 
perform various operations associated specifically with that 
menu or icon, such as obtaining or magnifying images, 
selecting particular parts of patient histories, etc. 15 

As shown in FIG. 2, the HUD display system 10 prefer- 
ably communicates with the command computer 36 via any 
appropriate form of wireless communication, as is well 
known in the art of computer networking. Thus, the surgeon 
is shown wearing a radio transmitter 44 and an antenna 46 
that operate in a manner that is weU known in the art to 
communicate with computer 36. Although it is preferred to 
use wireless communication, it is expressly noted that any 
and all forms of wireless or wired communication can be 
used, and as a result, the invention is not intended to be 
limited to any specific form of data communication between 
the HUD system and the computer 36. 

As discussed above, the surgeon may use the HUD system 
as a hands-free interface to control an imaging input device 
38 (shown as a camera in FIG. 2) and its associated 
manipulator assembly 39 to obtain images of the patient 34, 
and to display the images on the HUD display 14. In 
addition, the surgeon can operate the HUD system as an 
interface to obtain from computer 36 (or from additional 
remote or local computers not shown) reference or other 
patient data, and to display that data on the HUD display 14, 

Shown in FIG. 3 is a general block diagram of the HUD 
system integrated in a surgeon's environment in accordance 
with the present invention. The HUD system 10, including 4q 
the referenced display, eye-tracking, speech recognition and 
communication elements, is coupled to a main command 
control computer 36. Also coupled to the command com- 
puter 36 are the numerous sensor inputs 47, such as those 
that monitor vital signs and acquire medical images. An 45 
electronic data base 48 of the patient's history is either 
coupled to or included in the memory of the command 
computer 36, The command computer 36 has access over a 
standard network 50 to remote sites and computers (not 
shown). 5Q 

There are numerous types of sensor inputs 47 that will 
monitor the patient 34 and generate data of interest to the 
surgeon 12. Each such sensor 47 is considered old in the art, 
and operates to monitor and generate computer data defining 
the characteristics of the patient 34 in a manner well known 55 
to those of ordinary skill in the art. Thus, it is expressly noted 
that while several specific types of sensor inputs may be 
described in this specification, any and all types of sensor 
inputs 47 can be used, as long as the data is generated in a 
manner that is made accessible to the surgeon 12 by the 50 
command control computer 36. 

The patient data base 48 includes any and all type of 
patient data thai a surgeon may wish to access, including, for 
example, the patient's history and prior medical images. 
While the data base 48 is shown in FIG. 3 as being separate 6.S 
from the command computer 36, the data base can also be 
loaded into the data storage portion of the command com- 



puter 36. Likewise, the patient data base 48 may be located 
remote from the command computer 36, and can be accessed 
over the network 50. 

In its preferred form, the network 50 may access not only 
the standard hospital network, but also remote sites. In that 
manner, the surgeon 12 can access and communicate with 
other computers, expert systems or data devices (not shown) 
that are both local and remote from the surgeon's location. 
Likewise, specialists remote from the specific operating site 
may view the operation and communicate or consult directly 
with the surgeon 12, More specifically, the remote sites can 
selectively display the operation from any number of cam- 
eras in the operating room, and in addition, can selectively 
display and view the procedure with the same perspective of 
the surgeon 12 through the HUD display screen 14, using the 
head-mounted camera 29. In that manner, specialists at the 
remote sites will see what the surgeon 12 sees, including the 
view of the patient 34 and the data, menus, icons and cursor 
shown on the HUD display screen 14. In its preferred form, 
the video camera 29 is a remote controlled, high- 
performance camera that is mounted on the HUD gear worn 
by the surgeon 12 so that it can selectively view and transmit 
an image of the HUD screen 14 to the command computer 
36, and if desired, to a remote site or storage device (e.g., 
disk or video tape) controlled thereby. As shown in FIGS. IB 
and IC, the camera 29 can be mounted on the head of the 
surgeon 12 in a manner so as to make use of the same optics 
20 and 22 used by the display driver 18. In addition, as 
described below, the head mounted camera 29 and/or imag- 
ing device 38 may be mounted instead to the robotic arm 39 
controllable by the surgeon to tilt, pan, zoom or otherwise 
focus upon, selected views of the patient. The head mounted 
camera 29 may also be mounted other than on the top of the 
surgeon's head. For example, the camera 29 can be mounted 
on the left side of the surgeon's head, wherein additional 
optics are used to scan the display screen 14. 

FIG. 4 shows a more specific diagram of the main 
components of the HUD display system 10. The HUD 
system 10 is coupled to and communicates with the com- 
mand control computer 36 via a communication link 52. In 
its preferred form, the communications link comprises of 
two high speed digital radio transceivers or a optical com- 
munication system using fiber optic cable. The Hnk allows 
for video, audio, and/or data to be transported to and from 
a computer network to and from the operator in the form of 
graphics, audio, video, text, or other data. For examples of 
such systems, see R. Gagliardi et al., Optical 
Communications, (John Wiley & Sons, Inc. NY, 1995), C, 
Lynch et al., Packet Radio Networks: Architectures, 
Protocols, Technologies and Applications, (Pergamon Press 
NY, 1987), J, Cabral et al,, "Multimedia Systems for Tele- 
medidne and Their Communications Requirements," IEEE 
Communications Magazine (July 1996), M. Tsiknakis et al., 
"Intelligent Image Management in a Distributed PACS and 
Telemedicine Environment," IEEE Communications Maga- 
zine (July 1996), and A. Hutchison, "Electronic Data Inter- 
change for Health Care," IEEE Communications Magazine 
(July 1996). The above publications are incorporated herein 
by reference. It is noted that the communication Hnk 52 can 
use any method of communicating appropriate signals or 
information to and from a computer system and/or network 
(i.e. including but not limited to a radio transceiver, fiber 
optic cable, wire etc.). 

One use of the communication link 52 is to transmit to the 
command control computer 36 the input commands 54 
generated by the surgeon 12. The surgeon 12 generates the 
input commands 54 in one or more of several alternative 
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manners. Primarily, the commands 54 are generated when an to the eye-tracking system 56 and/or speech recognition 

eye-tracking system 56 detects the surgeon's eyes 32 focus- system 58, and displaying data and images obtained from the 

ing on selected icons or menu items displayed on the HUD numerous available sources. 

screen 14. The icons and menu items then cause the initia- Thus, the command computer 36 regularly 
tion of a corresponding operation, as is common with 5 communicates, through communication links 52, the control 
standard icon-based user interfaces employed with comput- signals necessary to operate the display driver or generating 
ers running the Macintosh or Windows 95 operating sys- system 18, which in turn creates and projects the required 
tems. Alternatively, the surgeon 12 may generate the com- elements of the basic user interface through the display 
mands orally by speaking select words or commands optics 20/22 onto the HUD screen 14. As the surgeon 12 
through microphone 16. A standard voice recognition sub- ^0 moves his eyes 32 to focus upon the various icons and 
system or computer 58 interprets the oral sounds output by menus of the user interface, the eye-tracking system 56 
the microphone 16, and generates digital commands 54 in generates input signals for the command computer 36, 
accordance with well known speech recognition processes. which in turn controls the display generating system 18 to 
These speech commands 54 are then passed to command correspondingly move the cursor 40 (shown in FIGS. 6 A, 
control computer 36 through communication links 52. For 15 6B, and 6C) on the display screen 14 in a manner that tracks 
more information on standard speech recognition systems, the movement of the surgeon's eyes 32. At the same time, 
see C. Schmandt, Voice Communication With Computers, the command computer 36 updates the status and contents of 
(Van Nostrand Reinhold, NY, 1994), and C. Baber et al, the various menus and icons, in a manner familiar to those 
Interactive Speech Technology: Human Factors Issues in the who use a "point-and-click" user interface, such as found in 
Application of Speech Input/Output to Computers, (Taylor 20 common Windows '95 and Macintosh computer systems 
and Francis, PA, 1993), incorporated herein by reference. using a mouse, touch-pad or similar device. As various 
For more information on programming icon or menu based menus and icons are selected, further input signals 54 are 
user interfaces, see J. Sullivan et ah. Intelligent User generated for use by the command computer 36, for 
Interfaces, (Addison-Wesley Publishing Company, NY, example, to obtain patient data or past or current images. In 
1991), incorporated herein by reference. 25 response, the command control computer 36 carries out the 
The communication link 52 is also responsible for routing required operations external to the HUD system 10 (such as 
video images from a camera and lighting system 49 config- controlling the sensor inputs 47 which may include imaging 
ured on the HUD system. More specifically, the camera and input device 38 or inputs &om data base information 48 or 
lighting system 49 generate video information under control other network 50 as shown in FIG. 3) to access and obtain 
of the surgeon 12 for display on the HUD screen 14. Thus, 30 tbe requested data. The command computer 36 then controls 
using commands generated by speech or from the icon/menu the HUD system 10 via communication Hnks 52 to update 
system, the surgeon controls pan driver 65, tilt driver 67, the screen 14 to display for the surgeon the requested data 
magnification driver 69 and light 31 to focus upon selected or images, and to generate audio through speaker 17. 
scenes for imaging. The pan driver 65 controls pivotal The display driver or generating system 18, shown in FIG. 
movement in the horizontal direction by the camera while 3s 4, operates to generate the images that are transmitted 
the tilt driver 67 controls the vertical pivotal scanning through the optics 20/22 and displayed on the HUD screen 
movement of the camera. The magnifier driver 69 controls 14. Such display drivers or systems are well known to those 
the degree of zoom of the image input device 38. The camera of ordinary skill in the art, and any applicable display system 
drivers each control a respective servomotor, stepper motor can be used. For example, it is known to use display 
or actuator that moves or controls the associated camera 40 generating devices such as CRTs, LEDs, laser diodes, LCDs, 
parameter. In that manner, the surgeon can control the etc., and this invention is not limited to any particular 
camera to focus upon and scan a particular feature (such as system, as long as it can generate and display video or 
a tumor), and to generate and display on the HUD screen 14 computer-generated images onto the display surface/screen 
highly magnified views thereof. In addition, the head 14 or directly into the user's eyes 32 thereby superimposing 
mounted camera 29 can be controlled to scan the HUD 45 images over the surgeon's actual field of view. See, for 
screen 14 to generate, record and transmit to remote sites the example, the following references related to display 
view as seen by the surgeon 12. Although only one head systems, each of which are incorporated herein by reference: 
mounted camera 29 is actually shown in the drawings, it A. Yoshida et al., "Design and Applications of a High- 
should be understood that multiple cameras can be used, Resolution Insert Head-Mounted-Display*', Proc. 95 
including multiple different types of cameras (such as video, 50 (pgs. 84-93, 1995), E. Williams, Liquid Crystals for Elec- 
television, infra-red), and that those and additional cameras tronic Devices (Noyes Data Corporation, NJ, 1975); M. 
may be controlled by other than the surgeon 12. Thus, the Tidwell et al, "The Virtual Retinal Display — ^A Retinal 
imaging input device 38 can either be controlled manually Scanning Imaging System," Proceedings of Virtual Reality 
and/or automatically. World '95 (pgs. 325-334, Munich, Germany: IDG Confer- 
In addition to routing input commands 54, eye vector 55 ences and Seminars, 1995); J. Kollin, "Optical Engineering 
information from eye-tracking system 56, and data from Challenges of the Virtual Retinal Display," Proceedings of 
image input device(s) 38 to the command control computer the SPIE (Vol. 2537, pgs. 48-60, Bellingham, Wash., 1995); 
36, the communication links 52 also serve to route control J. Kollin, "A Retinal Display for Virtual-Environment 
information from the command computer 36 to the HUD Applications," Proceedings of Society for Information Dis- 
system 10 to operate the various subsystems such as the 60 play (1993 International Symposium, Digest of Technical 
speaker 17, display driver 18, display screen 14, and imag- Papers, Vol. XXIV, pg. 827, Playa del Rey, Calif: Society 
ing input device 38. More specifically, the command com- for Information Display, 1993) and G. Robinson, "Display 
puter 36 operates to maintain the contents of the display on Prototype Uses Eye's Retina as Screen," Electronic Engi- 
HUD screen 14, including maintaining the display of the neering Times (pgs. 33-34, Apr. 1, 1996). 
basic menus and icons in accordance with the mode selected 6.5 In its preferred form, the HUD system 10 uses a projec- 
by the surgeon 12, controlling and displaying movement of tion method for displaying images in the user's field of view 
the ctu-sor 40 (shown in FIGS. 6 A, 6B, and 6C) in response using a light source (e.g. CRT, LED, laser diode, LCD 
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projector, etc.). The light source intensity or brightness can of view 72 of the user 12. More specifically, a surgeon 12 

be varied in the user's field of view so that the image being wearing the HUD system 10 will see the patient 34 on table 

displayed can be more visible than the surrounding light. 75 through the semi-transparent display screen 14. The user 

The light source intensity may also be decreased so that the will also see a number of icons or menu items 74, 76, 78, 80, 

surrounding light can become more visible and the image 5 82, 84 and 86, superimposed over the real scene along the 

being displayed less visible. top portion of the normal field of view 72. Alternatively, the 

Most CRT, LED, and other projection display methods icons or menu items 74, 76, 78, 80, 82, 84, and 86 can be 

require distance (optical path length) for projecting images. positioned along an opaque portion of the display screen 14, 

Thus, as shown in FIG. l,,a CRT/LED projector 18 is used outside the normal field of view 72. The specific contents 

as the display driver. In order to make the display screen 14 and form of the icon or menu items 74, 76, 78, 80, 82, 84, 

as small as possible, display mirrors 20 and 22 are used to and 86, along with their associated computer operations, will 

transmit the projected images to the screen 14. More vary depending on the specific implementation and pro- 

specifically, mirrors 20 and 22 are located within the head- gramming. However, in its preferred form, there will be 

mounted system 10, and are positioned outside of the field included icons or menu items that allow the user to control 

of view of the user 12. However, they reflect the projected one or more cameras, access one or more computers, control 

image so that it can be superimposed over a real scene on the the characteristics of the display 14, and display data, such 

display screen/surface 14 formed of a glass or other suitable as a patient's vital signs. 

display material. Another method of displaying images in For example, as shown in FIG. 6A, three separate icons or 

the user's field of view is by using LCD technology embed- menu items 74, 76 and 78, are assigned to control three 

ded inside the display surface 14. Here light from the 20 cameras, indicated as CAMl, CAM2 and CAM3, respec- 

surrounding environment is blocked or filtered by the dis- lively. By selecting any of these camera icons 74, 76, and 78, 

play when the appropriate voltage is applied to cells in a the user can independently control the associated camera 

LCD matrix. Part of the control of the display screen 14 is systems to obtain (pan, tilt, rotate, zoom, etc.) and display 

done through eye-tracking system 56. This eye-tracking various images. Similarly, two icons or menu items 80 and 

system 56 includes eye-tracking electronics 73, an infrared 25 82 are assigned to control access to two computers, indicated 

camera 26, eye-tracking optics 28, and an infrared laser 24, as COMPl and C0MP2, respectively. By selecting either of 

all of which are described below. the computer icons, the user can access and control the 

Shown in FIG. 5 is an example of an eye-tracking system associated computers or networks, to obtain patient data or 
56. Tliis system operates in a manner known to those of other reference material. Another icon or menu item 84 is 
ordinary skill in the art. Several such systems are readily 3Q indicated with the label DC and is provided to allow the user 
available, and the invention is not limited to any particular to access and vary the characteristics of the screen 14. For 
device, system, means, step or method for tracking the eye, example, the DC icon or menu item 84 can be accessed by 
For more detail on such eye-tracking systems, see for the user to control brightness, contrast, and the degree to 
example the following U.S. Pat. Nos. 5,231,674; 5,270,748; which you can see through the screen 14. Another icon 86 
5,341,181; 5,430,505; 5,367,315; 5,345,281; 5,331,149 and 35 allows the user to control various devices to obtain and 
5,471,542 incorporated herein by reference. In its preferred display on screen 14 any of a wide variety of vital signs, 
form, a low power laser 24 generates an infrared eye- As discussed above, each of the icons or menu items 74, 
tracking laser beam 60. The laser beam is projected through 76, 78, 80, 82, 84, and 86 can be accessed and controlled by 
a lens 62 and reflected by a mirror 28 onto the user's eye(s) causing the eye-tracking cursor 40 to move over and select 
32. The user's eyes include a sclera 64, cornea 66, and pupil 40 the desired icon. For example, referring to FIG. 6B, to see 
68. When the user's eye(s) 32 move, the eye components an update of the patient's vital signs, the surgeon can focus 
cause distortions in the infrared laser beam, which are his or her eyes 32 on the icon 86 corresponding to the 
reflected back onto mirror 28, and then through a lens 70 into patient's vital signs. The eye-tracking system 56 will track 
an infrared photodetector, infirared camera 26 or other type the surgeon's eyes 32 to cause the cursor 40 to scroll to the 
of photodetector. This distortion of the laser beam corre- 45 VITAL icon 86. Depending on the programming of HUD 
sponds to the eye direction vector which can be measured system 10, when the cursor 40 tracks over the VITAL icon 
accurately by eye-tracking electronics 73 (Shown in FIG. 4). 86, the patient's vital signs will be superimposed over a 
Data defining the eye direction vector is subsequently trans- portion of the surgeon's field of view 72. Shown in FIG. 6B 
mitted from the eye-tracking electronics 73 to the command is the display of the standard vital signs in analog graphic 
computer 36 through the communication links 52. For 50 format. Specifically, graphics are shown for the patient's 
calibration, the eye-tracking optics which include mirror 28, blood pressure 83, heart rate 85, respiratory rate 87 and body 
lens 62, infrared camera 26, and laser 24, may be automati- temperature 89. However, any additional vital sign (e.g., 
cally adjusted for optimal performance through the use of blood sugar, oxygen level, blood flow, etc.) can be pro- 
computer controlled actuators (not shown). grammed into the system and selected by the surgeon for 

It is expressly noted that, while separate eye-tracking 55 display. In addition to, or in place of, the analog displays 83, 

electronics 73 are shown in the block diagram as carried by 85, 87 and 89, digital values and titles can be displayed (not 

the heads-up display system 10, it is also possible to transmit shown). Likewise, the system can be programmed to display 

the raw data from the infrared detector imaging device 26 to the vital signs for a set period of time, continuously, or in an 

the command computer 36, which then determines the "on-ofP* fashion. 

associated eye direction vectors. Likewise, the eye-tracking 60 Referring now to FIG. 6C, there is shown the same view 

computer (and other electronics, if used) can be worn by the as FIG. 6B, with an image captured by an image input device 

surgeon 12 on a beU or backpack (not shown). 38 superimposed on the normal field of view 72. To select 

Shown in FIGS. 6 and 7 are the contents and arrangement a camera and display its image, the surgeon 12 focuses his 

of several preferred forms of visual displays for the screen eyes upon the associated icon, for example, the CAMl icon 

14, including exemplary icons and menus for the user 65 74. As above, the eye-tracking system 56 causes the cursor 

interface. Referring first to FIG, 6A, an object such a patient 40 to track to icon 74, and correspondingly initiates the 

34 is shown visible through the display 14 in the normal field desired camera image to be superimposed over the image of 
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the patient 34. In the example shown in FIG. 6C, the image camera is selected, the surgeon 12 can simply "hide" the 

is a 5-times magnified image of an incision 91 in the patient. image until it is again desired to view it, at which point it can 

ir desired, using appropriate menu icons, such as, for be easily selected. Thus, the surgeon is not required to 

example, the display control icon 84, the surgeon may also sequence through each of the menu levels to access the 

cause the display screen 14 to operate in an opaque mode, 5 desired image. For example, as shown in FIG. 7C, the user 

displayingonly the image of the incision 91 as if on a normal has configured and controlled the display to show the 

computer display screen. Likewise, the surgeon can magnify windows or regions having patient data 98, magnified cam- 

or otherwise control an image input device(s) 38 to obtain era view 100, MRI data 102, magnified skeletal view 96, and 

the image(s) desired, at the appropriate magnification. For whole body skeletal view 94. The surgeon 12 can indepen- 

example, FIG. 7A shows the display screen 14 operating in dently hide, re-size, and rearrange each of the windows, 

an opaque or semi-transparent mode with the incision 91 along with the transparency level of the screen 14, thereby 

magnified to a 10-times view. Also shown in FIG. 7A, the providing a maximum of flexibility. 

cursor 40 has been replaced with a cross hair sighting system The flexibihty of the system is further shown in FIG. 7D, 

formed by dotted lines 90 and 92, thereby allowing the which depicts the display 14 having various user selected 

surgeon 12 to precisely select portions of the image to be information, data, or images positioned thereon at varying 

still further magnified, enhanced, and/or centered in the ^^S^f ^ ^"t^^f ^ transparency. For example, in FIG. 

display. By way of further example, reference is made to ^^^^^^^ has placed in the center ol the display 14 a 

FIG. 7B, which depicts the display of skeletal FIGS. 94 and f'^''T'^'''TA'^'^^^^^ "^'^f^ 

1 I A 1. 1. .1 -t-^ ' .1 Ai\ field of View 72 displays the patient 34. When the surgeon s 

96 selected as alx>ve by the surgeon nmovmg the c^^^ 32 ^^^^J^^ the window 108, the cursor 40 

toslilIanolherofthecameraicons,forexample,CAM2,con ^^-^^^^^ j„ eye-tracking cross hairs 90 and 92. Under 

control of the eye-tracking system 56, the cross hairs 90 and 

If desired, the user interface and menu/icon programming 92 allow the surgeon 12 to select a specific portion of the 

can be configured to require the surgeon to take further patient 34, such as incision 91. The selected portion (e.g., 

action after the cursor 40 tracks over one of the icons. For part of the incision 91) of patient 34 is then locked on and 

example, and in the simplest form, once the surgeon causes 25 magnified as a separate view in a different window 112 in the 

the cursor to track over a selected icon, nothing may happen manner described above where a different set of selectable 

until the surgeon "clicks" a foot-operated mouse button (not cross hairs 115 and 117 are shown for further magnification, 

shown). In more complex forms of the invention, the sur- As also shown in FIG. 7D, data such as the patient name and 

geon can actually access the selected operation by tracking procedure description is displayed in the title portion 114 at 

the cursor to the selected icon and then speaking a select 30 the top of the display screen/surface 14. The surgeon 12 has 

code word (e.g., "select'* or "open") into microphone 16, selected more detailed patient data 116 to be displayed in a 

which word or words are interpreted by speech recognition separate window 116 in the lower left hand corner of display 

system 58. In still another form of the invention, the surgeon 14. Various medication listings 118 and recommendations 

can access the selected operation associated with a particular 120, along with current cursor or cross-hair coordinates 122 

icon or menu item by blinking a set number of times in quick 35 are displayed in a top portion of screen 14. Also selected for 

succession after tracking the cursor 40 to the desired located. display are programmable warning indicators 124, 126, and 

The blinking action is detected by the eye-tracking system 128, generally shown at the top right portion of the display 

56. screen/surface 14. The warning indicators may be pro- 

In its simplest form, the selection by the surgeon of a grammed by the surgeon 12 to monitor certain conditions 

specific icon will cause the associated data or images to be 40 and to visually indicate warnings at desired levels. The same 

displayed on the screen 14. In such a mode of operation, it programmable warning indicators will issue various levels 

is desirable to include numerous icons on the periphery of of audible warning tones to the surgeon 12 through the 

the field of view 72, so that the surgeon 12 can quickly select speaker 17. In the configuration of FIG. 7D, the surgeon 12, 

an operation. In a more complex form of the invention, a has selected and displayed the vital signs in a separate 

series of menus can be associated with each icon, each menu 45 window 130 at the top left corner of the display screen/ 

having successively more detail. For example, instead of surface 14, and a 3D model of a select portion of the patient 

having three camera icons 74, 76 and 78, a single "video" in window 132, which is continually updated, is shown 

icon can be substituted, which when selected by the cursor below the vital signs 130 and updated in real time. Other 

40, will change the display to then show the individual icons graphical information 110 is updated in real time and 

for each of the many available cameras. Next, when one of 50 displayed at the bottom right corner of the display screen/ 

the individual camera icons is selected by the cursor 40, the surface 14. Other pertinent images or data may be selec- 

display will again change to show individual icons for the lively displayed in other areas of the display 14, for example 

various camera controls, such to control the panning, tilting, skeletal images in area 131 and magnetic resonant imaging 

rotating, magnification, filters, manipulator members, etc. (MRI) data 132. All of the described operations can also be 

As indicated, in more complex forms of the invention, the 55 efiEected or initiated by using the speech recognition system 
HUD system 10 may incorporate a hierarchical icon or menu 

system, where several layers of menus are necessary to Thus, the overall medical HUD system 10 is extremely 

obtain and display desired data. In that case, greater flex- flexible, allowing each surgeon to customize the display and 

ibility is desirable in controlling how much data can be use only the features deemed appropriate, necessary and 

displayed on the screen 14 at any given time. More 60 desirable. In the simplest operating mode, the surgeon may 

specifically, as shown in FIG. 7C, in the more complex choose to display on a small part of the screen 14 only the 

forms of the invention, the well known programming tech- patient's vital signs, as shown in FIG. 6B. In other modes, 

niques from the Macintosh or Windows 95 operating sys- and depending on the procedure, the surgeon may elect to 

terns are adapted and included in command computer 36 to proceed in a semi-transparent mode, with several windows 

allow the display of multiple windows, that can be easily 65 of data or images, as shown in FIG. 7D. 

tiled, cascaded, selected, re-sized, or hidden in a manner Each of the variations in programming set forth above can 

now familiar to those skilled in the art. Thus, once a specific be configured for each specific surgeon 12 in a computer file 
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assigned to that user and stored in command computer 36. 
Thus, if a particular surgeon 12 prefers to have specific icons 
or menus shown on specific screens, or for example, prefers 
digital over analog displays for vital signs, that user can 
select those specific settings and the system will perform in 5 
accordance therewith. Similarly, the system allows for sub- 
stantial customization for specific types of surgeons or fields 
outside of surgery (e.g., microelectronics, forensics, etc.). 

Shown in FIGS. 8A, 8B, and 8C is still another embodi- 
ment of the invention that incorporates a non- attached jq 
heads-up display screen 134. In FIGS. 8A and 8B, the 
display screen/surface 134 is shown as a generally flat 
rectangular semi-transparent display area. The display 
screen/surface 134 is attached to a top end of a sturdy but 
flexible post 136 via joint 138, thereby allowing it to be 
moved to various viewable positions. The joint 138 is also 
constructed to allow the screen 134 to be moved left or right 
relative to the post 136. The lower end of post 136 is 
attached to a base 140 that is supported on the floor. The post 
136 is also adjustable in height and can be bent to allow 20 
precise positioning of the screen/surface 134 by the user. In 
a further modification to this non-attached HUD 
embodiment, as shown in FIG. 8C, the display screen 134 is 
mounted via robotic arm 142. Display screen/surface 134 of 
HUD 10 is attached at a lower end of the multi-jointed, ^5 
robotic arm or other manipulator 142. The upper end of arm 
142 is coupled to a mounting platform 144 that is fixed to the 
ceiHng 146 of a room. Speech commands such as "Adjust 
Display On", "Adjust Display Ofi", "Up", "Down", "Left", 
"Right", "Forward", and "Backward" can be used for con- 3Q 
trolHng the position of the retractable display screen/surface 
134. Here the position of the display screen/surface 134 may 
be robotically controlled by speech signals from an operator 
12. Such speech signals may be received by a microphone 
and processed by a speech recognition system which thereby 35 
sends signals to a robotic microcontroller that drives the 
appropriate actuators to position the display screen/surface 
134 as desired. 

In the embodiments of FIGS. 8A, 8B and 8C, the non- 
attached heads-up display system operates in a manner as 40 
described in the head-mounted HUD display embodiment of 
FIGS. 1-7, thereby allowing the surgeon to select and 
display data in a superimposed manner over the normal field 
of view of the patient 34. Both the speech recognition 
system 58 and eye-tracking system 56 can be used to move 45 
the cursor 40 to activate select icons or menus for operating 
computer 36 and display 134. However, in the modified 
forms of the invention shown in FIGS. 8 A, 8B and 8C, 
additional methods for moving the cursor are possible, 
including using a low power laser mounted on head of the 50 
surgeon 12, along with a touch screen incorporated in 
display screen itself. 

The foregoing description of a preferred embodiment and 
best mode of the invention known to applicant at the time of 
filing the application has been presented for the purposes of 55 
illustration and description. It is not intended to be exhaus- 
tive or to limit the invention to the precise form disclosed, 
and many modifications and variations are possible in the 
light of the above teaching. The embodiment was chosen 
and described in order to best explain the principles of the 60 
invention and its practical application, and to enable others 
skilled in the art to best utilize the invention in various 
embodiments and with various modifications as are suited to 
the particular use contemplated. 

For example, many computer-controlled instruments used 65 
in the micro-electronics field have one or more eyepieces 
through which the operator views a specimen. The preferred 



embodiment of the head.s-up display can be easily modified 
for use in such applications. More specifically, because the 
user's eye is relatively aligned with the eye-piece, the 
eye-tracking laser can be unobtrusively mounted in or below 
the eye-piece shaft. The user interface can be displayed by 
a display driver onto the same view seen through the 
eye-piece. In the same manner as described for the preferred 
embodiment, the user can see the fiill realm of data in the 
normal field of view, while simultaneously controlling the 
associated computer. Still further modifications are possible 
without departing from the spirit and scope of the invention. 

The HUD system's ability to provide high resolution 
images directly in the field of view of an operator without 
forcing the operator to look away can greatly enhance the 
abihty to complete an operation in a very precise and 
controlled manner. This precise control can be incorporated 
into surgical cutting, probing, and/or positioning tools by 
clearly presenting the position of such tools onto the display 
with respect to a patient and/or patient model obtained from 
real time imagery. This technique can be very advantageous 
in the event that the actual orientation or position of such 
tool(s) is unobtainable from an unassisted eye but requires 
direct visual control to operate. 

What is claimed is: 

1. A heads-up display system for use by a medical 
technician, the heads-up display system selectively control- 
lable by the medical technician comprising: 

a) a transparent heads-up display screen including a 
plurality of icons including a vital signs icon, the 
transparent heads-up display screen worn on the head 
of the medical technician, the transparent heads-up 
display screen positioned in the normal field of view of 
the medical technician; 

b) a command control computer coupled to the transpar- 
ent heads-up display screen and including an electronic 
storage device; 

c) a communication network coupled to the command 
control computer and further comprising: 

i. a wide-area network for accessing medical data bases 
to be accessed by the medical technician and for 
communication and consulting with other medical 
technicians; and, 

ii. signal transmitters and receivers for communication 
of information and images derived from the proce- 
dure for remote viewing, thus establishing a remote 
telepresence for consuhation, education or other 
medical purposes; 

d) an eye- tracking system integrated with the transparent 
heads-up display system, the eye-tracking system sup- 
ported on the head of the medical technician and 
coupled to the command control computer and 
configured, 

(i) to detect and track movement of one of the medical 
technician's eyes as the medical technician views the 
icons on the transparent heads-up display screen, and 

(ii) to communicate to the command control computer 
eye-tracking data corresponding to the movement of 
the medical technician's eye to the icons on the 
transparent heads-up display; and 

e) a user interface including: 

(i) an icon control program stored in the electronic 
storage device and configured to cause the command 
control computer to generate and maintain on the 
display screen a plurality of selectable control icons, 
including a vital sign icon in a manner that is 
superimposed over the normal field of view; 



us 6,847,336 Bl 



21 



22 



10 



15 



20 



(ii) a cursor control program stored in the electronic 
storage device and configured to cause the command 
control computer to: (1) generate and maintain on the 
display screen a moveable cursor in a manner that is 
superimposed over the normal field of view; (2) 
translate the eye-tracking data communicated from 
the eye-tracking system into cursor control data; (3) 
apply the cursor control data to cause the cursor 
displayed on the display screen to move in a manner 
that tracks movement of the medical technician's 
eye; and (4) generate process control data when the 
cursor is tracked over and selects the vital sign icon; 
and 

(iii) a process control program stored in the electronic 
storage device and configured to cause the command 
computer to initiate specific control actions corre- 
sponding to the process control data generated when 
the eye-tracking cursor is moved to the vital sign 
icon. 

2. The system of claim 1 wherein the heads-up display 
includes a display driver coupled to the command computer 
and configured so as to project images onto the display 
screen. 

3. The system of claim 2 wherein the command computer 

is coupled to the heads-up display by a radio communication 25 
link. 

4. The system of claim 3 wherein the command computer 
is coupled to the heads-up display by a wire communication 
hnk. 

5. The system of claim 2 comprising a voice recognition 30 
system including: 

a) a microphone placed proximate the user's mouth; 

b) a speaker placed proximate at least one ear of the user; 

c) a voice recognition circuit configured to recognize 
words spoken by the user and generate speech com- 
mand signals corresponding thereto; and 

d) wherein the process control program is configured to 
cause the command computer to initiate specific control 
actions corresponding to the speech command signals 
generated by the voice recognition circuit. 

6. The system of claim 1 wherein the icon control program 
generates and maintains on the display screen the vital sign 
icon relating to data acquisition procedures. 

7. A hands-free system for assisting medical practitioners 45 
in the performance of medical procedures on patients com- 
prising: 

a. a computer subsystem coupled to a patient medical data 
base subsystem, a communication network subsystem, 
a patient monitoring sensor subsystem, and a heads-up 
display (HUD) subsystem worn on the head of the 
medical practitioner; 

b. the computer subsystem further comprising: 

i. one or more computers programmed to respond to 
patient sensor inputs and to external HUD com- 55 
mands from medical practitioners; to retrieve and 
record medical data base information; to manage 
information flow to and from the external commu- 
nication network; to transmit medical images and 
information to the HUD; and, to perform analysis of 60 
information and present results of the analysis; 

ii. signal processing circuitry coupled to the computer 
subsystem for processing command signals and sen- 
sor information signals for input to the computer, and 

to process signals from the computer for audio, 65 
visual or other presentation to the medical practitio- 
ners; 
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iii. a first communication link for communicating with 
the heads-up display subsystem; 

c. the heads-up display (HUD) subsystem further com- 
prising: 

i. a second communication Hnk for communicating 
with the computer subsystem first communication 
link and with: (1) a HUD camera and light 
subsystem, (2) a HUD eye-tracking subsystem, (3) a 
HUD display subsystem, (4) a HUD audio input 
subsystem; and, (5) a HUD speaker subsystem; 

ii. the HUD camera and Hght subsystem coupled to the 
second communication link; 

iii. the HUD eye-tracking subsystem coupled to the 
second communication link; 

iv. the HUD display subsystem coupled to the second 
communication link; 

V. the HUD audio input subsystem coupled to the 

second communication Hnk; 
vi. the HUD speaker subsystem coupled to the second 

communication link; 

d. the medical data base subsystem coupled to the com- 
puter subsystem and further comprising: 

i. patient data base information including medical 
history, medical images, allergic reactions, family 
history and related data; 

ii. treatment information for various maladies and 
recommended procedures; 

iii. medication alternatives; and, 

iv. surgical recommendations for different patient con- 
ditions; 

e. the communication network subsystem coupled to the 
computer subsystem and further comprising: 

i. a wide-area network for accessing medical data bases 
to be queried by the medical practitioner and for 
communication and consulting with other medical 
practitioners; and, 

ii. signal transmitters and receivers for communication 
of information and images derived from the proce- 
dure for remote viewing, thus establishing a remote 
telepresence for consultation, education or other 
medical purposes; 

f . the patient monitoring sensor subsystem coupled to the 
computer subsystem and further comprising: 

i. vital sign sensors comprising blood pressure, heart 
rate, respiration rate, body temperature, or other 
patient status indicators; 

ii. signal generators coupled to the vital sign sensors for 
generating coded signals representative of the mea- 
sured vital signs; and 

iii. communication circuitry coupled to the signal gen- 
erators and the computer subsystem for formatting 
and transmitting sensor information to the computer 
sub-system; 

g. the computer subsystem being further programmed to 
operate the HUD to provide a user selectable display 
options comprising: 

i. patient images derived from the camera and lighting 
subsystem or from the computer data base 
information, including composite and overlay 
images, 

ii. sensor icons displaying sensor measurements from 
the patient monitoring sensor subsystems; 

iii. control icons or menus to control various sub- 
systems including the camera and lighting subsystem 
panning, tilting, magnification or zooming and light- 
ing; the display subsystem; and one or more com- 
puter subsystems; 
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iv. hierarchical icon or menu operation with layers of 
menus; 

V. multiple simultaneously viewable display windows 
arranged in tiled format with selected images, patient 
information, text information, or other related 
information, the display windows being semitrans- 
parent or opaque under user or under program 
control, individual windows being further selectable 
for expanded viewing, resizing, hiding, or 
rearrangement, 

vi. selectable, programmable warning indicators to 
visually indicate warnings at desired levels, 

vii. the display options being user selectable and con- 
trollable using the HUD eye-tracking subsystem by 
tracking the medical practitioner's eyes causing an 
eye tracking cursor to move over the display with 
user selection of the desired icon or menu by specific 
eye movements, a spoken command, or foot actua- 
tion of a system control to direct desired operations 
and displays. 

8. The invention in accordance with claim 7 wherein the 
first communication link of the computer subsystem further 
comprises: 

a wireless radio for transmission and reception of signals; 
an antenna coupled to the wireless radio; and 
signal routing and control circuitry for routing various 
signals to and from the computer subsystem. 

9. The invention in accordance with claim 8 wherein the 
second communication link of the HUD subsystem com- 
prises: 

wireless radio for transmission and reception of signals; 
an antenna coupled to the wireless radio; 
couplings to the HUD eye-tracking subsystem, display 
subsystem, audio input subsystem, camera and light 
subsystem and speaker subsystem for receipt and deliv- 
ery of messages and signals to, from and between these 
HUD subsystems; and, 
signal routing and control circuitry for routing various 
signals to and from respective subsystems. 

10. The invention in accordance with claim 9 wherein the 
HUD camera and light subsystem coupled to the second 
communication link comprises: 

at least one imaging device, such as a CCD camera, an 
x-ray scanner, a NMR scanner, a CAT scanner, a sonic 
scanner, or other medical imaging scanner, coupled to 
a pan driver, a tilt driver, a magnification or zoom 
driver and an illumination light for capturing images of 
selected areas of a subject patient for display, analysis 
or recording, the imaging device being further coupled 
to the second communication link for delivery of 
images of selected areas of the patient to the computer 
subsystem and to the HUD display subsystems; 

the pan driver coupled to the second communication link 
to receive control signals and further coupled to the 
imaging device to enable controllable panning of the 
subject patient with the imaging device to view differ- 
ent areas of the patient; 

the controllable tilt driver coupled to the second commu- 
nication link to receive control signals and further 
coupled to the imaging device to enable varying the 
viewing angle of the selected patient area with the 
scanning device 

the or zoom or magnification driver coupled to the second 
communication link to receive control signals and 
further coupled to the imaging device to enable con- 
trollable magnification of selected areas of a subject 
patient; and. 
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the light receiving control signals to provide controllable 
illumination to selected areas of a subject patient to 
enhance user or imaging device visibility. 

11. The invention in accordance with claim 10 wherein 
the HD eye-tracking subsystem coupled to the second com- 
munication link comprises: 

a low powered laser coupled to the eye-tracking electron- 
ics for generating an infrared eye-tracking laser beam; 

eye-tracking optics coupled to the eye-tracking electron- 
ics including a lens for focusing the laser beam onto a 
mirror positioned to reflect the beam onto the user's 
eye(s), and a mirror to positioned to capture reflected 
energy from the user's eye(s) and direct the reflected 
energy through a lens to an infra photo-detector cam- 
era; 

eye-tracking electronics coupled to the infrared photo- 
detector camera for detecting distortion of the reflected 
laser beam energy corresponding to the eye direction 
vector, and generating data defining the eye direction 
vector for transmission through communication links to 
the computer subsystem; and, 

a calibration subsystem coupled to the eye-tracking sub- 
system elements for adjtisting the elements for optimal 
performance through the use of computer controlled 
actuators. 

12. The invention in accordance with claim 11 wherein 
the HUD display subsystem coupled to the second commu- 
nication link comprises: 

a display screen or transparent or semi-transparent surface 
mounted in the field of view of the medical practitioner 
using the HUD for the display of images; 

display optics comprising mirrors or lenses for focusing 
of display images on the screen or surface for viewing 
by the medical practitioner, including capability for 
superimposing the display images over the real scene 
being viewed through the transparent or semitranspar- 
ent HUD viewing surface; and, 

a display driver coupled to the second communication 
link for receiving display signal and generating the 
actual display image to be viewed by the medical 
practitioner. 

13. The invention in accordance with claim 12 wherein 
the HUD audio input subsystem coupled to the second 
communication link comprises: 

an audio input microphone mounted on the HUD to 
enable the user to use spoken commands to direct 
overall system operation or to record comments; 

a speech recognition subsystem coupled to the audio 
microphone to recognize spoken commands and gen- 
erate control signals; and, 

an input command interpreter subsystem coupled to the 
speech recognition subsystem and the second commu- 
nication link to interpret commands from the speech 
recognition subsystem and generate computer com- 
mands for transmission to other system elements, 
including at least the computer subsystem, 

14. The invention in accordance with claim 13 wherein 
the HUD speaker subsystem coupled to the second commu- 
nication link comprises: 

a speaker mounted on the HUD to enable the user to hear 
spoken or otherwise audible commands generated by 
the computer subsystem or relayed by that subsystem 
from the external communication subsystem or infor- 
mation database subsystem or from the patient; and, 
speaker control circuitry to adjust the speaker volume or 
other audio parameters for optimum performance in the 
working environment. 
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[57] ABSTRACT 

The present invention provides a method and system for 
controlling content on a display of a computer system. The 
method and system comprises providing at least one control 
element on the display, moving the at least one control 
element selectively between a first position and a second 
position on the window. The method and system further 
includes the content between the first and second positions. 
The at least one control element allows the user to "fold" 
windows of information like a piece of paper, at any 
location, to bide information that is not of interest to the user. 
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METHOD AND SYSTEM FOR method and system comprises providing at least one control 

CONTROLLING CONTENT ON A DISPLAY element on the display, moving the at least one control 

SCREEN IN A COMPUTER SYSTEM element selectively between a first position and a second 

FIELD OF THE INVENTION position on the window. The method and system further 

^ . . , 5 includes hiding the content between the first and second 

Tbe present invention relates generally to a computer ^^^.^^^ j^^^^ ^^^^^^j ^jj^^^ 

system and more particularly to a method and system for ^^^j^,, ^.^^^^^ information like a piece of paper, at any 

controlling content on a display m such a computer system. , . . u j f +u + ■ ♦ • * * f »L 

^ ^ r / location, to hide information that is not of mterest to the user. 

BACKGROUND OF THE INVENTION 

Computers that include one or more windows on a display 10 BRIEF DESCRIPTION OF THE DRAWINGS 

screen have been in existence for more than a decade. ^ ^ ^^^^^ ^-^^ ^ ^.^^^^ information 

However, the amount of control that a user has over the * ■ j • i 

, . J f J- 1 t. L J contained in columns, 
content within the window of a display has changed very 

little. For example, it is known that a window on a display FI^' ^ ^? ^ ^^^^ ^^^^ ^ window showing information 

screen can be resized or repositioned. It is also known that 15 contained in rows. 

resizing a window usually results in sacrificing or removing FIG. 3 is a front view of a window showing a status area, 

content located at the bottom, top or sides of the window. In piG. 4 is a block diagram of a computer system, 

most applications there is very little control over what 5 ^^^^^ ^ ^.^^^^ ^ j ^ accordance with 

portion of the content is eliminated and the relationship of present invention 

that information that has been eliminated to the rest of the 20 , , ^ , ^ , ^ ^ 1 , 

content in the window, P^^- ^ ^^^^^ « ^^^^^ operation of a fold 

. , , 1 y control in accordance with the present invention. 

In addition, windows that are used in conventional appli- ^ 

cations do not allow for the display of noncontiguous or FIGS. 7-12 show in more detail the operation of the fold 

nonadjacent rows and columns. To further illustrate this control element when selectively hiding or showing columns 

problem, refer now to FIGS. 1 through 3. FIG. 1 illustrates 25 ^^^^^ window of a display, 
a window 10 on a computer display 11 which displays 
information in columns. As is seen, columns 1 through 7 

contain information in a column format. Referring now to x^e present invention relates to a method and system for 

FIG. 2, what is shown is a window 10' on the display 11' controlling content on a computer display screen. The fol- 

which displays content a row format through rows 1 through 30 lowing description is presented to enable one of ordinary 

10, FIG. 3 illustrates a window 10" which includes a status ^y^n the art to make and use the invention and is provided 

area 15 on a bottom portion of a window in which the the context of a patent application and its requirements, 

information is provided in rows and columns. As is seen. Various modifications to the preferred embodiment will be 

each of the windows 10, 10' and 10" of FIGS. 1, 2 and 3 ^adily apparent to those skilled in the art and the generic 

includes a titlebar 14 at a top portion of the window which 35 principles herein may be applied to other embodiments, 

includes control elements 16 and 18 which are traditionally tjjus, the present invention is not intended to be limited to 

used for a variety of functions. For example, the control the embodiment shown but is to be accorded the widest 

element 16 is used to move the window around the display scope consistent with the principles and features described 

screen and control element 18 closes the window. herein. 

Conventional windows do not aUow for flexibility when 40 4 ^ ^^^^^ ^.^g^.^^ ^ ^^^tem in which the present 

viewing content within the wmdow. Hence, for example, it invention resides. The system 50 includes a display 52, a 

is not possible to see column 2 and column 6 side by side keyboard 54, a pointing device 56, a processor 58, memory 

utilizing conventional applications. Similarly, referring now ^0, and a disk drive 62. The components of the system 50 are 

to FIG. 2, if it is desired to view rows 3 and 9 side by side, gjjown to be connected by a bus 64. 

it is not possible in conventional applications. Finally, 45 _ . . » ^ . 1 

J? • . .t- 1. f ■ . *i_ The present invention allows tor a user to control the 

refernng to FIG. 3, there may be a desire to remove the . / r -a i.- i_ • 1 j j- 1 c 

^ £.1 1, content of a window which is displayed on a display of a 

status area 15 because it serves no useful purpose, but once , , , j « - • J 

, /i r • computer system. In so doing, the user can review inrorma- 

aeain using conventional applications it can not be ehmi- . \ ^ i j • 

^^Q^ tion in nonadjacent columns or rows and m the appropnate 

* , . ^, , , ^^^^ . , , o cn circumstance eliminate unwanted information that is within 

In addition, although the FIGS. 1 through 3 show one 50 (jjgpiav 

window on a display, it is well recognized that a display ^ . ^ ^ . , .^^^ • 1 

screen associated with a computer system may have multiple Referrmg now to FIG. 5, a window 100 is shown on a 

windows thereon that are being viewed. It is desirable to display screen that includes a plurality of columns 1-7. In 

selectively remove content therein to minimize the usage of ^'1'''!'°" |°|he other control elements 16 , 16 and 18 the 

"real estate", i.e. space, on the display screen when viewing ss window 100 also includes a new control element referred to 

™ ..r;„A^..,^ as a fold control element 102. The fold control element 102 

multiple windows. , ,.11., . , 

. J,, * j*ujjr IS Utilized advantageously to create folds in the window uke 

Accordmgly, what is needed IS a system and method for . , 1 1 uvi / u * * 

, . 1 . ^ ^ a piece of paper to selectively hide/show content within the 

controlhng content on a display screen in a computer system , a- r ^ ^ ^ u *u 

^ , ^. ,■ . ^. f • • f *• display screen. To more particularly describe the use of this 

which allows for selective elimination of certain information , r , ,u e u - « • r 

therewithin while preserving the integrity of the information control, refer to the followmg figures m conjunction with the 

that is remaining. The system should be easy to use, should accompanying discussion. 

be adaptable to existing applications, and should be easy to "^^re clearly descnbe the operation of the fold control 

implement. ITie present invention addresses these needs. 102, refer now to FIG. 6, which is a flow chart of the 

operation of the flow control in accordance with the present 

SUMMARY OF THE INVENTION ^5 invention. First, it is determined if a fold is required, via step 

The present invention provides a method and system for 202. Then the next step is to determine whether the fold is 

controlling content on a display of a computer system. The horizontal, via step 204. If the fold is to be horizontal, then 
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a copy of a horizontal control element is moved to a first cancel button 118 is pressed, then the shaded area will be 

position on the display, via step 206, Next the control eliminated and the fold control will be reset. Until one of the 

element separates into a first and second portions, via step buttons are pressed on the set fold dialog box 114, the shaded 

208. Thereafter one of the first and second portions of the areas 108 and 112 can be increased or decreased in whatever 

horizontal control element is moved to a second position on 5 manner the user desires 

the display, via step 210, and finally, the content between the K^[^„ing „ow to FIG. 10, as is seen when OK is pressed 

first and second posjUons is hidden, via step 212_Next it is ^.^j ^ox 114. the fold is activated preferably with 

determined if the fold is to be removed via step 214. If it is ^ visual effect of moving inward asshownby the arro4 120 

not to be removed, return to step 216. If it IS to be removed, j « r • . i j i_ . 

^. * * i_ ^ n . J J ^- u ij and 122. Referring now to HG. 11, the marked area between 

then the content between the first and second portions should , « , ^ j ^ . , .\\ Jr . '7 ^ 7: ? v ;Xo 

cv 1 1 1 f 1^ • ♦ -i^A ,u columns 2 and 6 is now hidden. Only the dotted line 108 

reappear. Similarly, if it is a vertical fold, via step 214, the « . • j. . ^ j? i j a • i 

f. 1 PI, / 1 1 ♦ • i\ , r 1 remams to mdicate the presence of a fold. As IS also seen, the 

vertical lold control element is moved to a first vertical r i_ • . i ^ u .11 . • 

1 , M f ,u V, • * 1 copy of the honzontal fold control element 104 is one 

position on the display, via step 218. Next, the honzontal 1 . * • 1 .1. u . • . 1 j? ,j 

1 , . . ^ - 1 n . 1-1 element. As is also seen, the copy of the horizontal fold 

element is separated into first and second vertical portions, ! 1 i aTu ****'"_r''^"» . . • 

^t^« 11A vT^^t r.4^*ul control 104 has an X therewithm providmg a further indi- 

via step 220. Next, one or the first and second portions or the . -jjt lj- .t. ^ 

, _ . \ 1 ♦ 1 1 J, • ♦ 1 cationthatafoldisprovided. In a embodiment, the X can be 

honzontal control element is moved to a second honzontal , , f • ■ , 

position, via step 210, and then the content between the first P'^f ^"^^^ , . 

and second positions is bidden, via step 212. FinaUy, steps 3""* ^'''^^ 

214 and 216 are repeated as described above. allows one to compare nonadjacent columns of mformation. 

FIGS. 7-12 show in more detail the operation of the fold ,„ f^^^'^^ control element 104 and 

control element 102 when selectively hiding or showing f ,=°Py f^" ''Z^' by clicking or preying on the 

columns within the window of a display. Initially, as is seen, f , f '^f P'°'^!'l"^ 

in FIG. 7, the fold control element is pressed and in this be followed with the vertical fold control element 106 

embodiment two control elements 104 ind 106 are shown. '° ^ nonadjacent rows 

In this embodiment, control element 104 is utilized to „ Accordingly a system and method is provided that allows 

provide a horizontal fold. Control element 106 provides » « '° ^f'^. ''"h^" ^ ^^f^ow like a piece of 

vertical fold. First the horizontal fold control wiU be P»P«r selectively hide and reveal content Through a 

described system and method m accordance with the present 

. ' ■ciz-Qfi. f ,u .11™. invention, related nonadjacent information can be easily 

As IS seen in HG. 8, first a copy of the control element . . , , .... , . , , . . ..^ . 

104 is moved a predetermined distance. In this embodiment, 30 ^° """^f""' teal *«tate within a display of 

the fold is begun between columns 2 and 3, The movement f ?""Pf saved by ehminatmg unuseful 

of the copy of the control elements 104 and 106 can be "^0™^!'°"' a status bats and the Iflce. 

accomplished in a variety of fashions, for example, by using Although the present mvention has been described m 

a mouse (not shown) to drag the control to the right, using a«»rdance with the embodiments shown, one of ordinary 

some type of pointing device (not shown) or some other 35 ^^^^ ""^^^'^y ^cognize that there could be 

device to provide the beginning of the fold. As is also seen, variations to the embodiments and those variations would be 

in a preferred embodiment, there is a dotted line 108 which ^'•'i" «P'"' ^^d scope of the present invention. For 

will indicate the boundary of the area that is being hidden. example, although a copy of the vertical and horizontal fold 

When the initial move is completed, as is seen in the figure, elements 104 and 106 are utilized to create the fold, 

the copy of the fold control element 104 changes in appear- 40 " '^'^^^ ^^^^ elements could be used to 

ance by splitting in the middle, into a firet and second ^eate the folds Accordingly many modifications may be 

portions 104« and 104i>. ordinary skill m the art without departing 

„ c ■ . T7I/-I n • J . • J .u . .u froni the spirit and scope of the appended claims. 

Refernng now to FIG. 9, once it is determined that the , ■ 1 • j • 

beginning position of first and second portions 104fl and 1 a ^u * .• 1 . n- . . 

1046 is correct, then the first or second portion 104« and 45 «^^ly controlling content within a 

1046 can be moved to finish the fold. It should be recognized ^^"f ^^j, °" ' ^"^^^ °* " """P"'"' *y*""°' <=^V^«^% 

that the separation of the control element 104 into two f x ' . , , , , . , 

portions and the subsequent movement of one of the por- Providing at least one control element on the window; 

tions is an optional step. For example, the control element W separating the at least one control element mto first 

104 could be set at a first position and then by some user 50 second portions; 

interaction, such a mouse click, a first position of the fold is (c) moving one of the first and second portions to a second 

set. Thereafter the control element 104 could be set at a position on the window; and 

second position, then by a user interaction a second position (d) hiding the content between the first and second 

of the fold is set. positions. 

In this embodiment, the second portion 1046 is moved to 55 2. The method of claim 1 wherein the moving step (b) 

a second position between columns 5 and 6. As is also seen, includes the steps of: 

the dotted line 108 moves to a mid -point between the first (bl) separating the at least one control element into first 

and second portions 104^2 and 104^? . In a preferred embodi- and second portions; and 

ment there is a shaded area 110 indicated between columns (b2) moving one of the first and second portions to a 

2-6. In addition there is a shaded area 112 between the first 60 second position on the window. 

and second portions 104^ and 1046 of the control element 3. The method of claim 2 wherein the vertical fold control 

104. The shaded area 110 indicates the content to be hidden. element causes the content between a first and second 

When the second position is set, in a preferred embodiment vertical positions of the window to be hidden. 

a set fold dialog box 114 pops up. This dialog box 114 is 4. The method of claim 3 wherein the content between the 

utilized to indicate whether the fold is to be made or not 65 first and second vertical positions is shaded. 

within the window 100. Accordingly, in this embodiment, if 5. The method of claim 4 wherein the first position 

the OK button 116 is pressed, the fold can be made. If the includes a line for providing a fold position. 
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6. The method of claim 5 wherein the line moves based 
upon movement of one of a first and second portions of the 
vertical fold control element. 

7. The method of claim 6 wherein the line is dotted. 

8. The method of claim 7 wherein the fold position is 5 
based upon being between a raid-point of the first and 
second portions of the vertical control element. 

9. The method of claim 8 wherein a copy of the at least 
one control element is utilized to provide the fold. 

10. The method of claim 2 wherein the horizontal fold lO 
control element causes the content between the first and 
second horizontal positions of the window to be hidden. 

11. The method of claim 10 wherein the content between 
the first and second horizontal positions is shaded. 

12. The method of claim 11 wherein the first position 15 
includes a line for providing a fold position. 

13. The method of claim 12 wherein the line moves are 
based upon movement of one of a first and second portions 
of the horizontal fold control element. 

14. The method of claim 13 wherein the line is dotted. 20 

15. The method of claim 14 wherein the fold position is 
based upon being between a mid-point of the first and 
second portions of the horizontal control element. 

16. The method of claim 15 wherein a copy of the at least 
one control element is utilized to provide the fold. 25 

17. A system of selectively controlling content within a 
window on a display of a computer system, comprising: 

means for providing at least one control element on the 
window; 

means for separating the at least one control element into 

first and second portions; 
means for moving one of the first and second portions to 

a second position on the window; 
means for hiding the content between the first and second 35 

positions. 

18. The system of claim 17 in which the providing means 
comprises: 

means for providing a horizontal fold control element; 

and 40 
means for providing a vertical fold control element. 

19. The system of claim 18 wherein the vertical fold 

control element causes the content between a first and 
second vertical positions of the window to be hidden. 

20. The system of claim 19 wherein the content between 45 
the first and second vertical positions is shaded. 

21. The system of claim 20 wherein the first position 
includes a line for providing a fold position. 

22. The system of claim 21 wherein the line moves based 
upon movement of one of a first and second portions of the 50 
vertical fold control element. 

23. The system of claim 22 wherein the line is dotted. 

24. The vSystem of claim 23 wherein the fold position is 
based upon being between a mid-point of the first and 
second portions of the vertical control element. 55 

25. The system of claim 24 wherein a copy of the at least 
one control element is utilized to provide the fold. 

26. The system of claim 18 wherein the horizontal fold 
control element causes the content between the first and 
second horizontal positions of the window to be hidden. 60 

27. The system of claim 26 wherein the content between 
the first and second horizontal positions is shaded. 

28. The system of claim 27 wherein the first position 
includes a line for providing a fold position. 

29. The system of claim 28 wherein the line moves are 65 
based upon movement of one of a first and second portions 
of the horizontal fold control element. 



30. The system of claim 29 wherein the line is dotted. 

31. The system of claim 30 wherein the fold position is 
based upon being between a mid-point of the first and 
second portions of the horizontal control element. 

32. The system of claim 31 wherein a copy of the at least 
one control element is utilized to provide the fold, 

33. A method of selectively controlling content within a 
window on a dii^lay of a computer system, comprising the 
steps of: 

(a) providing a horizontal fold control element; 

(b) providing a vertical fold control element, wherein the 
vertical fold control element causes the content 
between a first and second vertical positions of the 
window to be hidden, wherein the content between the 
first and second vertical positions is shaded, wherein 
the first position includes a line for providing a fold 
position, wherein the line moves based upon movement 
of one of a first and second portions of the vertical fold 
control element; 

(c) moving the horizontal fold control element and the 
vertical fold control element selectively between a first 
position and a second position on the window; and 

(d) hiding the content between the first and second 
positions. 

34. The method of claim 33 wherein the line is dotted, 

35. The method of claim 34 wherein the fold position is 
based upon being between a mid -point of the first and 
second portions of the vertical control element. 

36. The method of claim 35 wherein a copy of the at least 
one control element is utilized to provide the fold. 

37. A method of selectively controlling content within a 
window on a display of a computer system, comprising the 
steps of: 

(a) providing a horizontal fold control element, wherein 
the horizontal fold control element causes the content 
between a first and second horizontal positions of the 
window to be hidden, wherein the content between the 
first and second horizontal positions is shaded, wherein 
the first position includes a line for providing a fold 
position, wherein the line moves based upon movement 
of one of a first and second portions of the horizontal 
fold control element; 

(b) providing a vertical fold control element; 

(c) moving at least the horizontal fold control element or 
the vertical fold control element selectively between a 
first position and a second position on the window; and 

(d) hiding the content between the first and second 
positions. 

means for hiding the content between the first and second 

positions. 

38. The method of claim 37 wherein the line is dotted. 

39. The method of claim 38 wherein the fold position is 
based upon being between a mid-point of the first and 
second portions of the horizontal control element. 

40. The method of claim 39 wherein a copy of the at least 
one control element is utilized to provide the fold. 

41. A system of selectively controlling content within a 
window on a display of a computer system, comprising: 

means for providing a horizontal fold control element; 

means for providing a vertical fold control element, 
wherein the vertical fold control element causes the 
content between a first and second vertical positions of 
the window to be hidden, wherein the content between 
the first and second vertical positions is shaded, 
wherein the first position includes a line for providing 
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a fold position, wherein the line moves based upon 

movement of one of a first and second portions of the 

vertical fold control element; 
means for moving at least the first or the second portions 

to a second position on the window; 
means for hiding the content between the first and second 

positions. 

42. The system of claim 41 wherein the line is dotted. 

43. The system of claim 42 wherein the fold position is 
based upon being between a mid-point of the first and 
second portions of the vertical control element. 

44. The syvStem of claim 43 wherein a copy of the at least 
one control element is utilized to provide the fold. 

45. A system of selectively controlhng content within a 
window on a display of a computer system, comprising: 

means for providing a horizontal fold control element, 
wherein the horizontal fold control element causes the 
content between a first and second horizontal positions 
of the window to be hidden, wherein the content 
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between the first and second horizontal positions is 
shaded, wherein the first position includes a line for 
providing a fold position, wherein the line moves based 
upon movement of one of a first and second portions of 
the horizontal fold control element; 

means for providing a vertical fold control element; 

means for moving at least the first or the second portions 
to a second position on the window; 

means for hiding the content between the first and second 
positions. 

46. The system of claim 45 wherein the line is dotted. 

47. The system of claim 46 wherein the fold position is 
based upon being between a mid -point of the first and 
second portions of the horizontal control element. 

48. The system of claim 47 wherein a copy of the at least 
one control element is utilized to provide the fold. 



liiiiiiiiiiiiiniiii 

US006983331B1 

(12) United States Patent (lo) Patent No.: us 6,983,331 Bi 

Mitchell et al, (45) Date of Patent: Jan. 3, 2006 



(54) SELECTIVE DISPLAY OF CONTENT 

(75) Inventors: Syne Mitchell, North Bend, WA (US); 

G. Scott Tomlin, BeUevue, WA (US); 
Lori Niva, Issaquah, WA (US); 
Matthew J. Lindenburg, Seattle, WA 

(US) 

(73) Assignee: Microsoft Corporation, Redmond, WA 
(US) 

( * ) Notice: Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C, 154(b) by 715 days. 

(21) Appl. No,: 09/691,277 

(22) Filed: Oct. 17, 2000 

(51) Int. CI. 

G06F 15116 (2006.01) 
G05F 15/00 (2006.01) 

(52) U.S. CI 709/246; 709/203; 709/217; 

715/513; 715/517 

(58) Field of Classification Search 709/217, 

709/203,246; 715/513,517 
See application file for complete search history. 

(56) References Cited 

U.S. PATENT DOCUMENTS 

6,023,714 A * ^2000 Hill et al 715/513 

6,098,085 A ♦ 8/2000 Blonder et al 715/513 

6,098,096 A * 8/2000 Tsirigotis et al 709/213 

6,237,030 Bl * 5/2001 Adams et al 709/218 

6,281,986 Bl * a/2001 Form 358/403 

6,300,947 Bl * 10/2001 Kanevsky 345/866 

6,336,124 Bl * 1/2002 Alam et al 715/523 

6,389,437 B2 * 5/2002 Stoub 715/523 

6,456,305 Bl * 9/2002 Qureshi et al 345/800 

6,462,752 Bl ♦ 10/2002 Ma et al 345/684 



6,539,406 Bl * 3/2003 Ibarra et al 715/513 

6,556,217 Bl * 4/2003 Makipaa et al 345/667 

6,593,943 Bl * 7/2003 MacPhail 345/734 

2004/0133848 Al * 7/2004 Hunt et al 715/500 

* cited by examiner 

Primary Examiner — ^Bharat Barot 

Assistant Examiner — David Lazaro 

(74) Attorney, Agent, or Firm — Ronald M. Anderson 



(57) 



ABSTRACT 



A method and system for selectively fully displaying addi- 
tional content in a browser window, without requiring scroll- 
ing in more than one direction, based on an available display 
area. For example, additional advertising content is thus 

selectively displayed in an otherwise unused area of the 
browsing window to the right of a primary content of a Web 
page. The size of the available display area is automatically 
detected, and a determination is automatically made as to 
whether the additional content and primary content can both 
be fully displayed in the available display area under the 
scrolling constraint. If so, both the additional content and the 
primary content are displayed, Otherwise, only the primary 
content is displayed. The additional content is downloaded 
only when it can thus be included within the display area. If 
the browser window is resized so that the additional content 
no longer fits without scrolling in more than one direction, 
the additional content is removed from the display, but is 
preferably retained in local storage for redisplay if the 
available display area is subsequently increased sufficiently. 
The type (and version) of browser program used is detected 
so that the appropriate instructions will be provided to the 
client device to selectively display additional content in this 
manner. For smaller client device displays, such as on those 
on pocket PCs, the content provided by a server is selec- 
tively determined in a prioritized manner, as a function of 
the available display area. 

6 Claims, 13 Drawing Sheets 
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SELECTIVE DISPLAY OF CONTENT 

FIELD OF THE INVENTION 

The present invention generally relates to a method and 5 
system for controlling retrieval and display of content on a 
display device, and more specifically, to selective retrieval 
and display of additional content within a limited browser 
window display area, without requiring scrolling. 

10 

BACKGROUND OF THE INVENTION 

World Wide Web (WWW) docunients, or Web pages, are 
widely used for distributing information to a variety of client 
devices, including personal computers. Web-compatible 15 
television set top boxes, personal data assistants (PDAs), 
wireless telephones, and other Internet connected devices. 
However, the amount, and quality of content that can be 
displayed in a Web page without scrolling depends on the 
size of the device's browser window, which is limited by the 20 
available display resolution. When only a small display is 
available, some of the content may be hidden until the user 
scrolls the display to disclose the content within the browser 
window. 

Clearly, scrolling to display hidden Web page content is 25 
not as convenient as displaying all of the content within a 
browser window. In contrast, when a large display area is 
available, the content often does not completely fill the 
available area, leaving blank space that could be employed 
to display additional content or information. This blank 30 
space within a browser window of a larger display area, 
which is not used for displaying the primary content of a 
Web page, would be especially useful for displaying adver- 
tising or other non-essential content. 

On most personal computers, the minimum display reso- 35 
lution is 640x480 pixels, but it is generally assumed that 
most users will set their computer display for a resolution of 
at least 800x600 pixels. Although the maximum display 
resolution of the video adapters and monitors used on many 
modern personal computers are substantially greater than 40 
this, Web content developers typically design Web pages to 
be fully displayable without horizontal scrolling in an area 
of 800x600 pixels. The additional area within the browser 
window on systems set to a resolution greater than 800x600 
pixels (when using the full display area) is thus typically 45 
unused on modern personal computer displays. Of course, a 
user can selectively set the size of a browser window on a 
display running at a higher resolution so that the browser 
window uses only a portion of the available display area. For 
example, the browser window can be selectively adjusted by 50 
a user for a width of 800 pixels on a display set for a 
resolution of 1024x768 pixels. 

Conversely, on devices having display areas that are 
smaller than 800x600 pixels, such as PDAs or Internet 
enabled cell phones, a user will be required to scroll both 55 
horizontally and vertically to view all of a conventional Web 
page width. Clearly, it would be preferable to supply a Web 
page to a display of an Internet connected device that is 
appropriate for the currently available browser window area, 
so that the content can be viewed without scrolling in at least 60 
one of the horizontal and vertical directions. 

Unfortunately, server-side documents and applications 
have no way of directly detecting the amount of display area 
available in the browser window on any given client device 
at any particular time. There are only a few ways of 65 
adjusting the amount of display area available on any given 
client device, running any given browser program, at any 
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given time. For example, it might be possible to force a new 
browser window to open, and specify the browser window 
size to accommodate the Web page being requested. How- 
ever, in this case, it would be necessary to make an initial 
assumption about the display area size available on the chent 
device and predetermine the window size of the Web 
browser program. In any case, this solution would likely not 
be acceptable to users, because it would require the 
requested Web page to change the browser window size, 
which most users would likely find objectionable. The more 
common solution to this problem is for the server to simply 
download a requested Web page in the size required to hold 
additional content. However, scroll bars are then required to 
display the additional content, if the browser window on the 
client device is not sufiBciently large. 

In some instances, pre -designed content is split into layers 
or frames that draw from multiple source files. Each layer is 
set to be either displayed or hidden. However, there is no 
known use of layers or frames to selectively download 
and/or display portions of content based on fitting the 
content within the available display area to limit scrolling to 
at most one direction. Layering was designed to allow 
portions of content to overlap each other or appear in a 
sequence to provide an animation effect. Layering was not 
designed to provide non-overlapping selective display of 
content modules based on the size of the display area, 
because scrolling is relied on for viewing any content that is 
beyond the margins of the display area. Note also that 
layering is only applicable to controlling a browser program 
display, usually on a client device. Layering is not applicable 
to controlling an active server page, or other server program 
that does not relate to displaying content. Such server 
programs usually relate to non-visual data processing and 
communicating content from the server. Alternatively, 
frames enable a display to be split into sections, but each 
frame is provided with its own scrolling capability. Also, any 
additional scroll bars required in a frame take up more 
display area. Thus, frames do not enhance efficient use of the 
display or limit scrolling to at most one direction. 

For many smaller devices with fixed display areas, servers 
detect the requesting device and provide separate specialized 
content that is formatted for the small di.splay area. The 
content may be an HDML, WML, or other document format 
optimized for the size of the display of the requesting device. 
Often a separate single page of content is pre-defined to fit 
the display of the numerous possible requesting devices, 
which requires that multiple documents formatted for each 
display size be provided. 

Alternatively, some systems use proxy servers with con- 
tent manipulators to pre-process existing documents to 
reduce, convert and/or compress the content before sending 
it to the small device. Such systems include Puma Tech- 
noloy, Inc.'s PROXYWARE™, International Business 
Machines, Inc.'s transcoding proxies, and Spyglass, Inc.'s 
SPYGLASS PRISM'^M _ jhe approach used by these systems 
is effected on proxy servers between the Web server con- 
taining the requested document and the requesting client 
device and provides for translating existing documents to a 
format suitable for the requesting device having a small 
display. However, no provision is made in these systems for 
the original designer of the content to modularize or priori- 
tize the content, to control the portions of the content that are 
displayed based upon the available display or browser 
window area. 

A further alternative would be to scale some content to fit 
within an available display area on a client device. However, 
this solution may cause content designed for a small display 
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to be enlarged to fit within a large display area so much as additional content was previously retrieved, it is obtained 

to present a poor impression. Alternatively, content origi- from the local storage and displayed along with the primary 

nally designed for a high resolution display may be reduced content, when the display area becomes sufficient to meet 

so much to fit in a relatively small display as to be unread- the requirements regarding scrolling in no more than one 

able or unusable. Also, graphical advertisements, such as 5 direction. Also, if the browser window is reduced so that the 

banner advertisements, are not typically dynamically resiz- additional content cannot be displayed in the available 

able, and their effectiveness will likely be substantially display area, as noted above, the additional content is 

reduced if they are scaled. A better solution would be to removed from the display, but remains available in the local 

determine the available size of the browser window or the storage if again needed, should the display area again 

display (if all of the display is available) and send only the lO become sufficient to include the additional content, 

highest priority portion of the content that will fit the Anotheraspectof the invention is directed to a method for 

available display area without requiring scrolling in more selectively displaying a selected portion of content (without 

than one direction, or more preferably, without requiring a distinction between primary and additional content). An 

scrolling in other than a vertical direction, or most prefer- appropriate portion of the content is selectively displayed, 

ably, without requiring any scrolling. 15 based upon the available display area. In one preferred 

embodiment, portions of the content are assigned a prc- 

SUMMARY OF THE INVENTION defined priority, to determine the portions that are displayed, 

as a function of the available display area. Alternatively, the 

The present invention addresses the problems identified content is altered to fit within the available display area by 
above by selectively displaying portions of content to fit an 20 selecting those portions whose combined sizes make maxi- 
available display area, instead of leaving some display area mum use of the available display area, 
unused or requiring a user to selectively view all the content Yet another aspect of the invention is directed to a method 
by scrolling. A first aspect of the present invention is for selecting portions of content document based on a known 
directed to a method for selectively displaying additional available display area, before conveying the selected por- 
content on a display device within a limited browser window 25 tions to a browser program. This method is especially useful 
display area. The method includes the steps of detecting the for pre-selecting, communicating and displaying appropriate 
size of an available display area, and determining whether portions from within a page to fit small fixed-displays used 
the additional content and primary content can both be fully for browsing, such as on some PDAs and cellular phones, 
displayed in the available display area without requiring Another aspect of the invention is directed to a computer- 
scrolling in more than one direction. If so, both the addi- 30 readable medium having computer-executable instructions 
tional content and primary content are displayed. Otherwise, for performing the method disclosed above, A further aspect 
only the primary content is displayed in the display area. of the present invention is directed to a system that includes 

The available display area includes the portion of a elements that carry out functions generally consistent with 

browser window that is available for displaying content (i.e., the steps of the method discussed above, 

not occupied by tool bars, status bars, or other graphical user 35 

interface elements of a browser program). The method BRIEF DESCRIPTION OF THE DRAWING 

includes the step of detecting properties of the browser FIGURES 
program, such as the specific browser program and version, 

so that appropriate instructions can be provided to the j^e foregoing aspects and many of the attendant advan- 

browser program that are compatible with it and will enable 40 tages of this invention will become more readily appreciated 

detection of the dimensions of the display area, ^s the same becomes better understood by reference to the 

The step of determining whether both primary and addi- following detailed description, when taken in conjunction 

tional content can be fully displayed in the available display ^^^^ accompanying drawings, wherein: 

area in this manner occurs when primary content is initially ^ ^ schematic block diagram of an exemplary 

loaded by the browser program, whenever the browser 45 personal computer (PQ system suitable for implementing 

window is re-sized, or upon the occurrence of another ^j^^ present invention- 

predefined event. Tlie method can be implemented as an is a representation of a browser window display- 
event handler that takes_ advantage of browser event triggers . ^ ^^^^ ^^^^^^^ ^ 
and will then store a pointer to any previously defined event v^t^ -iti- t u j j i 
handler used by the browser program that is replaced or 50 . FIG- 2B is a representation of a b^^^^^^^ 
supplemented by the event handler of the present invention, ^" additional content to the right of the primary content; 
If appropriate, such a previously defined event handler is 2C is a representation of a browser wmdow display- 
then executed after the selective display of the additional ^n additional content displayed to the left of the primary 
content, or if only the primary content is displayed. content; 

Preferably, the method determines if at least one dimen- 55 is a representation of a browser window display- 

sion (e.g., a width or a height) of the available display area ^ plurahty of additional content selections, in two 

is at least equal to a predefined value, so that the display area columns to the right of the primary content; 

is sufficient to display the additional content without scroll- FIG. 2E is a representation of a prior art browser window 

ing in more than one direction. The method can also in which the size of the display area is such that it can only 

iteratively determine whether still more content can be fully 60 display a primary content of a Web page and which employs 

displayed in the available display area without requiring scroll bars to display any additional content; 

scrolling along with the primary content and any previously FIG. 3 is a flow diagram illustrating the logic used by the 

added content. Preferably, scrolling should not be required present invention to provide instructions that are compatible 

in a direction extending across the primary content and all of with a requesting browser to selectively display additional 

the additional content in the available display area. 65 content in a Web page based on the available display area; 

The additional content is retrieved only when it can be FIG, 4 is a block diagram illustrating components used by 

displayed and is then retained in local storage. If the a preferred embodiment of the present invention to provide 
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instructions that are compatible with a requesting browser to routines that helps to transfer information between elements 

selectively display additional content in a Web page based within the personal computer 20, such as during start up, is 

on the available display area; stored in ROM 24. The personal computer 20 further 

FIG. 5 is a flow diagram illustrating the logic used by the includes a hard disk drive 27 for reading from and writing 

present invention to selectively display additional content in 5 to a hard disk, not shown, a magnetic disk drive 28 for 

a Web page based on the available display area; reading from or writing to a removable magnetic disk 29, 

FIG. 6 is a flow diagram illustrating the logic used by the and an optical disk drive 30 for reading from or writing to 

present invention to selectively display multiple content a removable optical disk 31, such as a CDROM or other 

modules in a Web page based on the available display area optical media. Hard disk drive 27, magnetic disk drive 28, 

and a prioritization assigned to the content modules; 10 and optical disk drive 30 are connected to system bus 23 by 

FIG. 7 is a flow diagram illustrating the logic used by the a hard disk drive interface 32, a magnetic disk drive inter- 
present invention to selectively display multiple content face 33, and an optical drive interface 34, respectively The 
modules in a Web page based on the available display area drives and their associated computer readable media provide 
and the size of the content modules; nonvolatile storage of computer readable machine instruc- 

FIG. 8A is a representation of a browser window on a tions, data structures, program modules and other data for 

typical PDA with a plurality of content modules displayed; personal computer 20, Although the exemplary environment 

FIG. 8B is a representation of a browser window on a described herein employs a hard disk, removable magnetic 

small cellular telephone with a plurality of content modules disk 29, and removable optical disk 31, it will be appreciated 

displayed; and ^^°se skilled in the art that other types of computer 

FIG. 9 is a flow diagram illustrating the logic used by the 20 readable media, which can store data that is accessible by a 

present invention to pre-select and display multiple content computer, such as magnetic cassettes, flash memory cards, 

modules of a Web page based on a known display area and ^^^^^^^ "^^^^^ ^^s^^' Bernoulh cartndges, random access 

a priority of the content modules. memories (RAMs), read only memories (ROMs), and the 

like, may also be used in the exemplary operating environ- 

DETAILED DESCRIPTION OF THE ^5 ment. 

INVENTION ^ number of program modules may be stored on the hard 

disk, magnetic disk 29, optical disk 31, ROM 24 or RAM 25, 
including an operating system 35, one or more application 

Exemplary Operating Environment programs 36 (such as a browser program), other program 

FIG. 1 and the following discussion are intended to 30 modules 37, and program data 38. A user may enter com- 
provide a brief, general description of a suitable computing mands and information into personal computer 20 through 
environment in which the present invention may be imple- input devices such as a keyboard 40 and a pointing device 
mented, both in regard to a server that stores and provides 42. Other input devices (not shown) may include a micro- 
Web pages and a client that requests the Web pages and phone, joystick, game pad, satellite dish, scanner, or the like, 
displays them to a user. Although not required, the present 35 These and other input devices are often connected to pro- 
invention will be described in the general context of com- cessing unit 21 through an input/output (I/O) interface 46 
puter executable instructions, such as program modules that that is coupled to the system bus. The term I/O interface is 
are executed by a personal computer. Generally, program intended to encompass each interface specifically used for a 
modules include routines, programs, objects, components, serial port, a parallel port, a game port, a keyboard port, 
data structures, etc. that perfonm particular tasks or imple- 40 and/or a universal serial bus (USB). A monitor 47 or other 
ment particular abstract data types. Moreover, those skifled type of display device is also connected to system bus 23 via 
in the art will appreciate that this invention may be practiced an appropriate interface, such as a video adapter 48, and is 
with other computer system configurations, particulariy in usable to display Web pages, and/or other information. In 
regard to a client device for displaying a Web page, includ- addition to the monitor, personal computers are often 
ing hand held devices, pocket personal computing devices, 45 coupled to other peripheral output devices (not shown), such 
digital cell phones adapted to connect to a network and other as speakers (through a sound card or other audio 
microprocessor based, or programmable consumer elec- interface — not shown) and printers, 
tronic devices, multiprocessor systems, network personal Personal computer 20 may operate in a networked envi- 
computers, minicomputers, main&ame computers, and the ronment using logical connections to one or more remote 
like. The invention may also be practiced in distributed 50 computers, such as a remote computer 49. Remote computer 
computing environments where tasks are performed by 49 may be another personal computer, a server (which is 
remote processing devices that are linked through a com- typically generally configured much like personal computer 
munications network. In a distributed computing environ- 20), a router, a network personal computer, a peer device, 
ment, program modules may be located in both local and satellite or other common network node, and typicafly 
remote memory storage devices. 55 includes many or all of the elements described above in 

With reference to FIG. 1, an exemplary system for imple- connection with personal computer 20, although only an 

menting the present invention includes a general purpose external memory storage device 50 has been illustrated in 

computing device in the form of a conventional personal FIG. 1. The logical connections depicted in FIG. 1 include 

computer 20, provided with a processing unit 21, a system a local area network (LAN) 51 and a wide area network 

memory 22, and a system bus 23. The system bus couples 60 (WAN) 52. Such networking environments are common in 

various system components including the system memory to offices, enterprise wide computer networks, intranets and the 

processing unit 21 and may be any of several types of bus Internet. 

structures, including a memory bus or memory controUer, a When used in a LAN networking environment, personal 

peripheral bus, and a local bus using any of a variety of bus computer 20 is connected to LAN 51 through a network 

architectures. The system memory includes read only 65 interface or adapter 53. When used in a WAN networking 

memory (ROM) 24 and random access memory (RAM) 25. environment, personal computer 20 typically includes a 

A basic input/output system 26 (BIOS), containing the basic modem 54, or other means for establishing communications 
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over WAN 52, such as the Internet. Modem 54, which may Invention, it is undesirable for a user to scroll both vertically 

be internal or external, is connected to the system bus 23, or and horizontally to access all of a Web page — primary and 

coupled to the bus via I/O device interface 46, i.e., through added content. 

a serial port. In a networked environment, program modules FIG. 2A illustrates a generally conventional browser 

depicted relative to personal computer 20, or portions 5 window 60 with a content display area 62, a menu and 

thereof, may be stored in the remote memory storage device. toolbar area 64, a status bar 66, and a scroll bar 68. Shown 

It will be appreciated that the network connections shown ^^^^i^ content display area 62 is a primary content 70. 

are exemplary and other means of establishing a communi- P"°^^y ^^"tent 70 may comprise text, video or graphics 

cations link between the computers may be used, such as "^^y .^^^^^^ "^'^f"^' content of the various 

wireless communication and wide band network links. ^^P^ ^^P^^^^^y f,^^^^^"^^ P^gf • page docu- 

ments are typically requested by a user from a remote server 

Exemplary Implementation of the Present Invention ("^ot shown) and are downloaded from the remote server into 

-pu p n • J u T • 1 * * • ' the browser window over the Internet, and/or other network. 

The foUowmg descnbes an exemplary implementation m , . * . . 1 auv4/wi wi w uvi^ wiiv, 

r;, r T u * 4^ *u *' such BS mtranct. Most Web page documents are designed 

a first preferred embodiment of the present invention cor- , , . j. , j ... • j /s j j- i T. 

J. • J- 1 ■ J 1 , . 15 to be displayed withm a predefined display area, such as 
responding to its use m displaying additional content gen- 800x600 pixels. However, the browser x^dndow may be 
eraUy to the nght of the prmiary content m the display area ^^^^ maximized (or adjusted to a larger size than the 
of a browser window on a personal computer monitor. ^^^^ ^^^^^ ^^^^^ ^^^^^^^ ^^^^^^^ ^^^^p^) 
Additionally, the foUowmg implementation is described in ^ display or monitor having a substantially greater resolu- 
the context of a browser program displaying Dynamic ,0 tion, e.g., 1024x768 pixels. When primary content 70 is 
Hyper-Text Markup Language (DHTML) Web page docu- displayed within this larger area of browser window 60, an 
menls on a client computer in communication with a server additional display area 63 is available within content display 
via the Internet. However, as suggested eariier, this example area 62, but is not used for the primary content. In most 
is not meant to be limiting, as it will be understood by those cases, additional display area 63 is thus "wasted." Con- 
skilled in the art that the concepts and method disclosed 25 versely, if the primary content is designed for a display area 
below are applicable to any type of device for displaying larger than display area 62, a user must scroll the primary 
content, including network appliances, PDAs, cellular content within display area 62 using scrollbar 68 (and/or its 
phones, and the like. It is also contemplated that the concepts horizontal equivalent — not shown), to view the remainder of 
and method disclosed below are applicable to any type of primary content, 

documents based on a markup language, including HTML, enhance the user experience when viewing a Web 

standard generalized markup language (SGML), extensible P^g^» ^ desirable to reduce or eliminate the need to scroll 

markup language (XML), hand-held device markup Ian- ^he entire content that can be displayed. The user 

guage (HDML), wireless markup language (WML), and f ^^^^^ ^ '^""^^^ f "'^^^ ^^^^^^^^^ 

other documents of the like. It will further be understood that (S>foMy. only vertically) to fully view a Web page. On the 

the concepts and method disclosed below are applicable to desirable for commercial reasons to 

- ., J . ^ , , . J. maximize the amount of content displayed in content display 

any type of compiled or rnt^reted oompone^ mcluding ^^^^ particularly if the additional content added to the 

Microsoft Corporation sWINDOWS™ scnpt components. web page includes advertising that can increase revenues for 

component object model components, document object ^ ^-^^ 

model components, Microsoft Corporation's ACTIVEX™ To balance these competing interests, a preferred embodi- 

controls and Active Server Pages, common gateway mter- j^ent of the present invention selectively displays added 

face applications, Sun Microsystems, Inc's JAVA™ applets, content 72 in an unused portion 63 of content display area 

.TAVA™ servlets, and the like. Also, the concepts and 62, as shown in FIG. 2B. In the preferred embodiment of the 

method disclosed below are applicable to any type of present invention shown in FIG. 2B, unused portion 63 of 

compiled or interpreted computer code language, including 45 content display area 62 is disposed to the right of primary 

C++, Sun Microsystems, Inc.'s JAVA™, JavaScript, content 70, and added content 72 will Hkely (although not 

VBScript, ECMAScnpt, WMLScnpt, Peri, and the like. necessarily) include advertising that is not generally related 

Generally, this embodiment of the present invention to the primary content. However, different preferred embodi- 

selectively adds extra content to make use of otherwise ments of the present invention can alternatively display 

unused available space in a browser window display area, 50 added content 72 in other portions of the display area 62, 

where the available space is beyond the limits of a primary such as to the left of the primary content 70, as shown in 

content in at least one direction. But the extra content is FIG. 2C. Furthermore, the added content may be directed to 

displayed only if the available area in the window (or information that is related to the primary content, 

display) is sufiBciently large to accommodate both the pri- As additionally illustrated by FIG. 2D, multiple additional 

mary and the additional content without requiring the user to 55 content can be selectively displayed in the unused portion of 

scroll in more than one direction. Preferably, any scrolling available content display area 62. Moreover, this embodi- 

should be in a direction that does not extend across the ment of the present invention selectively displays both 

primary and extra content and will typically be only in the primary content 70 and additional content 72 in content 

vertical direction, since in many cases, the extra content will display area 62 only when the primary content and the one 

be selectively added to one side of the primary content. 60 or more modules of additional content can be displayed 

Simply adding additional content to an existing Web page without requiring scrolling in more than one direction. Thus, 

without regard to the available display area in a browser in FIG. 2D, added content 72 and added content 72* would 

window, as commonly done in the prior art, is an undesirable be displayed along with primary content 70 only if there is 

alternative to the present invention, because a user may be sufiScient display area to the side of primary content 70 so 

required to scroll both vertically and horizontally to view all 65 that all three modules can be displayed across the width of 

of the content, i.e., both the primary content and the addi- the screen without scrolling both vertically and horizontally, 

tional content. As noted above in the Background of the In this example, scrolling is required only in the vertical 
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direction to enable all of the contents of these three modules one direction. This feature eliminates the need to again load 

to be fully displayed, because the primary content extends the additional content from a remote permanent storage, 

below the bottom of the available content display area. which would likely be on a server. 

As a further alternative, the added content 72 could be Preferably, the present invention is automatically initiated 

displayed to the left of primary content 70, and added 5 when a Web page is initially loaded, and in response to a 

content 72' displayed to the right of the primary content 70. resizing of the browser window for any reason. These events 

As a still further alternative, either or both of the added are triggers to which the browser program will typically 

content modules could be displayed above or below the already respond, and can be utilized to initiate browser 

primary content 70. While less desirable, scrolling would compatible instructions that are different than those nor- 

still only be necessary in the vertical direction to display all lo mally executed by the browser program. One embodiment 

of the primary and added content. The invention can also be of the present invention includes an event handler that 

practiced to selectively display additional content only if preemptively extends any corresponding previously defined 

scrolling will not be required in either direction. event handlers. A Web page designer simply includes a call 

The following example should ensure that the differences to the invention, which automatically responds to each 
between the present invention and the conventional 15 relevant event trigger, such as the initial loading of the Web 
approach for handling content greater than an available page or the resizing of the browser window, 
display or browsing window area is clear. In the conven- The specific event handler instructions used to implement 
tional approach, scrolling in both the vertical and horizontal the present invention depend upon the type and/or version of 
directions may be required to display all content if a browser browser program used to display the Web page content. FIG. 
window 60 is limited or resized to such dimensions that 20 3 illustrates how specific browser-compatible instructions 
content display area 62 is insufBcient to display either or are provided to the browser program. In this preferred 
both the primary and additional content in both directions. embodiment, a browser program executing on a personal 
FIG. 2E illustrates a prior art browser window 60' that is computer connected to the Internet communicates via Trans- 
sized so that only primary content 70 can be fully displayed mission Control Protocol/Internet Protocol (TCP/IP) with a 
across the width of the content display area. Phantom lines 25 Web server. When a user of the browser program requests 
indicate where a portion of additional content 72 would content from the server by selecting or entering a uniform 
appear if it were added without employing the present resourcelocator (URL) for a Web page, the browser program 
invention, and it should be understood, that unless the user creates a request header, as indicated in a block 100 of FIG. 
scrolls to the right, the remainder of the additional content 3. A request header is typically a Hypertext Transfer Proto- 
that is currently hidden will not be displayed. Employing the 30 col (HTTP) string that identifies the client device and its IP 
present invention, the browser program would not even load address on the network. The request header also identifies 
additional content 72, because to do so would require that certain properties of the requesting browser, such as the type 
the user scroll both vertically and horizontally to view the of browser program and its version. The type of browser 
entire content. By detecting that insufBcient display area may be Microsoft Corporation's INTERNET 
dimensions exist during an initial load of the primary 35 EXPLORER™, Netscape Corporation's NAVIGATOR™, 
content of a Web page, the present invention eliminates the or other browser that operates on one of the various client 
unnecessary communication and retrieval of that additional devices described above. The portion of the header that 
content, and avoids the undesired scrolling that would be includes the browser type is often referred to as the "user 
required to fully view additional content 72. agent string." At a block 102, the browser program com- 

Clearly, a user may reduce the size of a browser window 40 municates the header, along with the rest of the content 

after additional content 72 has already been fully displayed. request to a control object in software executed on a Web 

The size of the browser window is typically reduced by server. 

pressing a restore/maximize button 74, by dragging a The request is typically for a Web page document con- 
browser window dynamic resize control 76, or by dragging taining the primary content 70 at a Uniform Resource 
an edge of browser window 60. In response to these actions, 45 Locator (URL) address. Preferably, the requested Web page 
the browser program would normally add horizontal and/or is an active server page that contains instructions to execute 
vertical scroll bars 68 (if not akeady provided) as needed to a control object on the Web server that will provide the 
enable the user to view all of the content in each direction browser compatible instructions. Another alternative would 
in which the full content is no longer displayed. Browser be to intercept the request at some intermediate point prior 
programs are not currently designed to remove any content 50 to retrieving the requested Web page and pre-process the 
from the display in an attempt to reduce or eliminate the request to identify and obtain browser compatible instruc- 
need for scrolling. In fact, they are specifically designed to tions. Yet another alternative would be to intercept the 
automatically provide scrolling, because they currently treat request and pre-determine whether and which additional 
all content equally. To provide greater control over the content 72 (if any) should be returned to the browser 
display of content, the present invention responds to actions 55 program along with primary content 70. 
that reduce the size of the browsing window below that In the illustrated preferred embodiment, the requested 
required to display all of the content without scrolling, by Web page calls a control object on the server that reads and 
removing additional content 72 from the display, so that at parses the request header to obtain the browser type and 
most, primary content 70 must be scrolled to view all of it. version information as shown at a block 104. The parsed 
As indicated above, removing additional content 72 to 60 browser type and version information is compared with 
reduce or eliminate scrolling of the browser window in more known browser types (and versions — although not specifi- 
than one direction improves a user's experience. Also, the cally shown) in decision blocks 106, 108, and 110, etc. to 
additional content, once downloaded to be displayed, is determine the instructions that will be compatible with the 
preferably retained in memory to enable the additional browser program that sent the request. At one of blocks 112, 
content to quickly be re-displayed if the browser window is 65 114, 116, etc. the compatible instructions for the browser 
subsequently sufficiently enlarged to fully display the pri- program (and version) that was determined are appended to 
mary and additional content without scrolling in more than the requested primary content Web page file. In a preferred 
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embodiment, these instructions are script code, such as modified page is then communicated back to the browser on 

JavaScript, that are appended to the Web page document the client device over the Internet or other network that 

containing the requested primary content 70. At a block connects the server with the client device. 

1118, the Web server control object then sends the modified As described above, the instructions included in the 

Web page to the browser program on the client device over 5 modified Web page override prior event handlers but pre- 

the network. serves a pointer to the prior event handlers for later execu- 

When the modified Web page is received by the client at tion, as appropriate. The core selective display operation is 

a block 120, the browser program first executes that portion executed as a superceding event handler by the client 

of the new instnjctions that overrides or supercedes prior browser as illustrated in FIG. 5. As also noted above, this 

corresponding event handler instructions, if any, which were lO event handler embodiment is triggered by an OnLoad, 

included in the original Web page for responding to a trigger Resize, or other pre-selected event, 

event, such as an OnLoad event or a Resize event, Specifi- Upon the occurrence of such an event, a decision block 

cally, the new instructions first rename any event handlers 170 in FIG. 5 provides for determining whether enough 

previously assigned to an OnLoad, Resize or other prede- display area is available in the browser window to display 

termined event trigger. This step preserves the prior event 15 both primary and additional content without scrolling in 

handlers for later execution. Then, the new instructions more than one direction. Browser-compatible instructions 

assign the new event handler to these event triggers. At a call on Document Object Model components of the browser 

block 122, the browser program on the client device then program to detect the dimensions. For example, JavaScript 

executes the new instructions comprising the superceding instructions compatible with Microsoft's Internet Explorer 

event handler to selectively display additional content if the 20 browser call a "clientWidth" property of the Document 

display area is sufficient to support both the primary and object to determine the width in pixels of content display 

addifional contents without requiring scrolling in more than area 62 in the browser window. 

one direction. If the size of the content display area is insufiBcient to 

FIG. 4 illustrates aspects of this preferred embodiment of display the added content and the primary content without 

the present invention in regard to the control objects 25 requiring scrolling in more than one direction, remaining 

executed on the Web server. As indicated above, a requested display portion 63 around the primary content is cleared in 

Web page document 130 preferably contains both primary a block 172, and by following the pointer to the prior event 

content 70 and script code. However, it may contain only handlers, they are executed, as provided in a block 180. 

code that refers to another primary document file, or files Thus, in an initial OnLoad case, the new event handler 

that contain primary content 70. Preferably, the requested 30 instructs the browser program to forego even downloading 

Web page document does not include additional content 72. additional content 72 if the display is not sized sufficiently 

Instead, additional content 72 is preferably contained in a to accommodate (i.e., view all of) both the primary content 

separate document file. Thus, requested Web page 130 is and additional content without scrolling in more than one 

preferably an active server page (ASP), Common Gateway direction. This feature ehminate the unnecessary download- 

Interface (CGI) appHcation, or other functional document. 35 ing of additional content 72 from the Web server. Alterna- 

Such a document will contain JavaScript, Perl, or other lively, in response to an OnResize event, clearing the 

executable code embedded within it. Those skilled in the art remaining display portion 63 effectively removes additional 

will recognize that requested page 130 is not limited only to content 72 from the display when content display area 62 is 

Internet Web pages, but may be any type of file or other reduced to a size insufScient to fully display both primary 

resource that can be requested by a client device for display. 40 and additional content without scrolling in more than one 

When a browser program executed on a client device direction . Thus, the same instructions are effective for mul- 

requests Web page 130 from the Web server, the script code tiple types of event triggers. 

within the requested page 130 makes a function call 132 to Conversely, if sufficient content display area 62 is avail- 

a selective display control object 134. Selective display able, a decision block 174 provides that the event handler 

control object 134 contains a control function or control 45 determines whether the additional content was previously 

method 136 that controls the Web server side operations. Ma downloaded, which may have occurred in order to display 

a data retrieval operation 138, control method 136 retrieves the additional content before the browser window was 

client request header information 140 and communicates the reduced in size. If the additional content was not previously 

information through a function call 142 to a browser detec- downloaded, then a request is made to the server to down- 

tion sub-object 144. Browser detection sub-object 144 50 load additional content 72 at a block 176. Waiting until this 

implements a browser detection method 146 that parses and point to download the additional content avoids unnecessary 

detects the browser properties from client request header time in downloading the additional content if it will not be 

information 140. The browser detection method then relays displayed. However, once the additional content has been 

the browser type and version via a function call return 148 downloaded and displayed, saving the additional content in 

back to control method 136, Control method 136 uses the 55 a cache in memory on the client device avoids the time 

browser type and version to determine the compatible required to download the addifional content again. Once the 

browser instructions that should be appended to requested additional content is available at the client device, at a block 

Web page 130, Control method 136 makes an appropriate 178, the browser displays the additional content along with 

function call 150 or 150', respectively directed to MS the primary content. 

Method 152 or Netscape Method 154 (or other), to the 60 A second preferred embodiment of the present invention 

appropriate method to retrieve or generate instructions to employs layer or frame concept when displaying content as 

selectively display additional content in the indicated a function of the available browser window/display size, 

browser program. The appropriately called method retrieves This embodiment first determines whether enough display 

or generates code compatible with the browser program area is available to display the additional content without 

indicated to be running on the client device. The compatible 65 requiring scrolling in more than one direction, and if so, 

code is communicated via a function return 156 and added generates or otherwise provides appropriate browser com- 

to requested page 130 as browser compafible script 158. The patible instructions for displaying a layer that includes the 
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additional content. The additional content source file is be accomplished by reading DIV tags or individual style 

communicated to the browser program along with these attributes of a markup language document. This embodi- 

instructions. In all other respects, this second embodiment is ment is especially well-suited for implementations that 

identical to the first disclosed above. Those skilled in the art utilize cascading style sheets and/or other forms of absolute 

will recognize that the approach used in this second embodi- 5 positioning. As before, a block 222 detects the browser type 

ment can be iterated for selectively displaying multiple and version and returns corresponding browser compatible 

modules of content so long as sufficient content display area instructions along with the parsed size data for the content 

62 remains to display each successive module. However, if modules. A block 224 indicates that a primary content 

the display area is only sufficient for the primary content, module is displayed. An alternative variation of this embodi- 

then no browser compatible instructions are transmitted to lo ment displays the largest content module that will fit within 

the client device or executed for displaying a layer of the available content display area without scrolling in more 

additional content. Also, no source file of the additional than one direction (or optionally, without scrolling at all), 

content layer is communicated to the browser program. A decision block 226 determines whether there is enough 

Once the additional content is displayed in either the first or display area remaining for the smallest currently non-dis- 

the second embodiment, the pointer to the corresponding 15 played content module to be added. If not, then optionally, 

prior event handlers, if any, enables them to be executed at the smallest non-displayed content module is modified in an 

step 180 in FIG. 5. attempt to fit the module to the remaining display area at a 

FIG. 6 illustrates logic for another iterative embodiment block 227. Modifications include changing one or more 
of the invention. The iterations in this case are based on a dimensions of the content module, scaling the content mod- 
priority assigned to content modules. In a block 200, a 20 ule, changing text font size within the content module, 
priority is assigned and the size of each content module is rotating the content module, providing a moving image of 
defined. The size is preferably defined in pixels. As in the the content within the content module, converting the con- 
previous embodiments, the browser type and version are tent to audio or other manipulation of the content module, 
determined from the header of a request received from the These manipulations can be accomplished with a variety of 
client device in a block 202, and browser compatible instruc- 25 methods, including changing individual style attributes 
tions are prepared for communication to the browser pro- within tags, block attributes, or other content attributes. The 
gram being executed on the client device. However, in this changes can also be accomplished dynamically with script- 
embodiment, the content module priorities and sizes are also ing. Once a modification is made, it is determined at optional 
communicated to the browser program. In a manner analo- decision block 229 whether the modified content module 
gous to displaying primary content 70, this embodiment 30 will now fit in the remaining display area. If so, then the 
displays a first (highest) priority content module in a block process can continue attempting to fit more content modules. 
204. In a decision block 206, the browser compatible Otherwise, the remaining display area is again cleared in a 
instructions determine a remaining display area and compare block 208, and the pointer that was saved for any prior event 
that remaining area with the size of the next priority content handlers is used to execute the prior event handlers as 
module. If the remaining display area is insufficient to 35 provided in block 180. If enough display area remains for 
accommodate the next priority content module (i.e., the display of at least the smallest currently non-displayed 
content module with the next lower priority), then this content module, then the logic of this embodiment attempts 
embodiment clears the remaining display area at the margin to determine the largest possible remaining content module 
of the display (or browser window), as indicated in a block that can be fit into the remaining display area. Initially, the 
208. As in previous embodiments, the pointer to the prior 40 logic assigns the smallest non-displayed content module as 
event handlers corresponding to those implemented by the a selected content module in a block 228. In a block 230, the 
instructions transferred to the browser program, if any, is logic identifies the size of the next larger currently non- 
recalled and the prior event handlers are executed in block displayed content module. The size of the next larger content 
180, just as in the previous embodiments. module is compared with the remaining display area to 

If the determination made in decision block 206 is that 45 determine whether this next larger content module can be 

sufficient display area is available to accommodate the next inserted into the remaining content display area, in a deci- 

lower priority content module, a decision block 210 deter- sion block 232. If so, a block 234 identifies this next larger 

mines whether the next priority content module has been content module as the selected content module instead of the 

already downloaded. If not, the next priority content module previous smaller content module, and the logic loops back to 

is downloaded to the browser program in a block 212. This 50 block 230, The loop that includes block 230, decision block 

next priority content module is then displayed in a block 232, and block 234 continues reiteratively, until the largest 

214. If the next priority content module was already down- content modules not currently displayed that will fit in the 

loaded in the determination made in decision block 210, it remaining content display area is identified. Each iteration 

is also displayed as provided in block 214. The logic then could also optionally include block 227, or sub-iterations of 

returns to decision block 206 to determine whether any 55 block 227, to modify the content module in further attempts 

remaining display area can accommodate the next lower to fit the module to the remaining display area. Correspond- 

priority content module. This iteration continues until the ingly, an optional decision block 229 can be included to 

remaining display area is insufficient in size to accommodate determine whether the modified content module will fit. The 

the next priority content module. step of modifying individual content modules can be incor- 

The logic for another embodiment is illustrated in FIG. 7. 60 porated into any of the embodiments to fit the content into 

This embodiment maximizes use of the available display the display area or otherwise provide the content (e.g. 

area by selecting and displaying content modules that will provide an audible content), subject to the display area 

fill the largest amount of display space available without restraints. A decision block 236 then determines whether the 

requiring scrolling at all, or alternatively, in no more than selected content module has already been downloaded. If 

one direction. To do so, this embodiment parses the content 65 not, a block 238 provides for downloading the selected 

modules for their dimensions and/or designated positions in content module. As before, once the browser program has 

a display or browser window, in a block 220. This step can recalled the selected content module from a storage (if it was 
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previously downloaded and displayed) or newly down- other on-line services often must be scrolled into view, 
loaded the selected content module, then the selected con- However, it will be apparent that selectively downloading 
tent module is displayed in the remaining content display and divSplaying only icons from a content document, without 
area in a block 240. Although the largest possible content the text, will enable more content items to be displayed, and 
module has thus been selected, some additional content 5 may also result in unused content display area being avail- 
display area may still remain in which a smaller content able. 

module may be displayed. Thus, the logic returns to decision Accordingly, yet another preferred embodiment of the 

block 226 to reiterate the process until not even the smallest present invention facilitates the download and display of 

currently remaining non-displayed content module will fit selected portions of a content document as a function of 

into the remaining content display area. lo content display area 62 in a browser window 260 (or full 

As noted above, this embodiment does not rely on know- display area) on a client device. In this embodiment, the 

ing the size of a primary content. It need not assume, for content that is downloaded to the client device might be an 

example, an 800x600 pixel resolution. Instead, it dynami- entire Web page if the client device is a personal computer, 

cally determines the size of each content naodule and or may comprise a selected one or more content modules 

dynamically positions the largest currently non-displayed 15 comprising the Web page if the client device that is detected 

content modules that will fit, in the available content display is a cellular phone or other client device having a very small 

area. display. 

However, some browser programs, particularly older ver- FIG. 9 illustrates the logic employed in this embodiment 
sions of currently popular browser programs, do not have the to selectively download and display one or more prioritized 
ability to dynamically detect the size of an available content 20 content modules, A priority is pre-assigned to each content 
display area. In other cases, the browser window size can not module available for display of a full Web page, in a block 
be changed and may be matched or limited to the entire 280. A content module may include only individual display 
physical display area provided on the client device, as will elements, if desired. As in earlier embodiments, the type and 
typically be true of reduced-capability browser programs version of the browser program executed on the client 
that execute on client devices such as PDAs, cellular tele- 25 device are detected in a block 282, preferably from a user 
phones, and other similar small, portable client devices. agent string included in a request header transmitted from 
Nevertheless, many of these browser programs will still the client device to a Web server. However, this step also 
have the ability to display content in various locations within detects the device type, preferably from the header. With this 
the content display area, even if the content is limited to a information, browser compatible instructions and the avail- 
few small icons or only a few lines of text. Thus, some 30 able display area are determined by the Web server control 
portion of the display area on such client devices may still object in a block 284. As discussed above, the available 
be unused, or if the present invention is not employed, content display area is determined based upon the known 
display of all of the content may require scrolling in more area of the requesting browser type and/or client display 
than one direction. For such client devices, selective loading device. In a block 286, a priority content module is com- 
and display of content modules in accord with the present 35 municated to the browser program with associated browser 
invention can still be accomplished on initial content load- location coordinates and other attributes, if any. In a pre- 
ing, although no client-side browser window resize-detec- ferred embodiment of this logic, determination of known 
tion handler is required. available content display area and selection of priority 

FIG. 8A is a representation of a browser window 250 that content is done on Web server before any content module is 
might appear on a PDA, running Microsoft Corporation's 40 communicated to the requesting browser program running 
POCKET INTERNET EXPLORER'^" browser program. on the cUent device. This step eliminates the need to 
The browser window includes content display area 62, with dynamically detect the available contact display area via the 
additional available display area 63 remaining after primary browser program DOM. A block 288 determines the area for 
content 70 is displayed. Just as explained above in connec- the next priority content module, preferably from the style 
tion with the larger browser windows employed on personal 45 attributes of that content module. Because the areas of each 
computers, the total display area on this PDA is limited by content module, and the available content display area are 
including tool bar 64 and any scroll bar 68 that is necessary known, the remaining content display area can be computed 
for viewing content beyond what will fit at one time in on the Web server to determine whether the next lower 
content display area 62. However, browser window 250 can priority content module will fit within the remaining display 
not be resized, so its content display area 62 is fixed and 50 area, as indicated in a block 290. If so, then that next lower 
known. Nevertheless, a plurality of content modules, such as priority content module is transmitted to the browser pro- 
primary content 70 and additional content modules 72 and gram. These steps iterate until insufficient content display 
72', can be selectively downloaded and displayed, based on area remains to display any added content. This embodiment 
the known browser window/display size. may also be readily adapted to send all selected content 

FIG. 8B is a representation of a browser window 260 on 55 modules during one communication from the Web server to 

a cellular telephone. Content display area 62 on this client the browser program, instead of during iterative downloads, 

device includes only text 268 and/or icons 262. Content As a specific example of the preceding embodiment, a 

display area 62 of small client devices like this cellular Web designer can assign higher priority to icons than 

phone may also have scroll direction indicators 264 con- associated text, so that all icons are sent to a cellular 

trolled by buttons (not shown). Such small client devices 60 telephone before text. Those skilled in the art will recognize 

also often have an "Enter" button (not shown) to select a that position attributes of the text content are not lost, 

highlighted content item 266 on the display. A content item because absolute or relative position attributes can be 

may comprise a URL hyper link to a Web page with assigned separately to the text as well as to the icons. Thus, 

additional detail about the content item, such as a list of text would be included in the appropriate locations for larger 

parameters, information, or other data that can thus be 65 browser windows, e.g., on a PDA, but not included for small 

selectively downloaded and displayed. For example, details browser windows on smaller client devices such as cellular 

and data relating to sports news, directions to a business, or telephones that don't have enough display area to accom- 
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modate the lower priority content text. Those skilled in the 
art will recognize that content can be selectively displayed 
based on size, or other attributes in addition to, or instead of 
priority. In any case, icons and/or other content can be 
selectively downloaded and displayed as a function of the 
available display or browser window, without requiring a 
separate Web page document for each different size of client 
device or different size of browser window. Web clipping, or 
other proxy server content conversion is not required on the 
client device, because the content is already prioritized. 

Although the present invention has been described in 
connection with the preferred form of practicing it and 
modifications thereto, those of ordinary skill in the art will 
understand that many other modifications can be made to the 
present invention within the scope of the claims that follow. 
Accordingly, it is not intended that the scope of the present 
invention in any way be limited by the above description, 
but instead be determined entirely by reference to the claims 
that follow. 

The invention in which an exclusive right is claimed is 
defined by the following: 

1. A method for controlling content displayed as a single 
Web page on a device, where the device is incapable of 
displaying all available content, comprising the steps of: 

(a) automatically determining an available display area 
for displaying the single Web page on the device; 

(b) automatically determining a combination of different 
portions of the available content that can be fully 
displayed in the single Web page in the available 
display area without requiring scrolling in more than 
one direction and without requiring a modification of 
the different portions of the content that results in the 
different portions comprising the combination being 
perceived in less than their entirety on the single Web 
page; and 

(c) displaying only the combination of different portions 
of the content that were automatically determined to 
enable a user to discern all of the information that the 
combination of different portions of the content pro- 
vides as fully displayed in the available display area, 
since the combination of different portions of the 
content does not employ a hyperlink to display the 
information it conveys, without requiring scrolling in 
more than one direction, and without requiring a modi- 
fication of the combination of different portions of the 45 
content that results in the different portions of content 
comprising the combination being perceived in less 
than their entirety. 
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2. The method of claim 1, further comprising the step of 
dividing the Web page into discrete content modules that are 
selectively display able. 

3. The method of claim 1, wherein the step of determining 
the portion of the content that can be fully displayed in the 
available display area without requiring scrolling in more 
than one direction comprises the step of prioritizing different 
portions of the content based on an order of importance, said 
different portions of the content corresponding to specific 
areas of the Web page that are selectively displayed as a 
function of the available display area. 

4. llie method of claim 3, wherein the step of prioritizing 
comprises the step of ranking the different portions of the 
content according to their importance in conveying infor- 
mation on the Web page. 

5. The method of claim 1, wherein the step of automati- 
cally determining the combination of different portions of 
the content that can be fully displayed in the available 
display area without requiring scrolling in more than one 
direction, comprises the steps of: 

(a) detecting a respective share of the available display 
area required by each portion of the content; and 

(b) automatically determining a combination of different 
portions of the content that are displayable in the 
display area, based upon the respective share of the 
display area required by each different portion of the 
content. 

6. The method of claim 5, wherein the step of automati- 
cally determining the combination of different portions of 
the content comprises at least one of the steps of: 

(a) automatically changing a dimension of a portion of the 
content to fit the dimensions of the available display 
area; 

(b) automatically scaling a portion of the content to fit the 
available display area; 

(c) automatically changing an orientation of a portion of 
the content to fit the available display area; 

(d) automatically converting a portion of the content into 
a moving image within the available display area; and 

(e) automatically subdividing a portion of the content into 
sub-portions that are displayed sequentially within the 
available display area. 
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ABSTRACT 



A universal interface apparatus for use with any of several 
different imaging systems for facilitating a plurality of 
different imaging modalities, the interface including at least 
function icons in a function navigation space and a 
workspace, each of the function icons corresponding to a 
process which is common among all of the imaging 
modalities, the interface also accessing data tables corre- 
sponding to specific workflow protocols for medical facility 
radiologists and/or for a medical facility in general, the 
tables identifying function icons and other icons for guiding 
a technologist through a properly orchestrated imaging 
process. 
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MULTIPLE MODALITY INTERFACE FOR 
IMAGING SYSTEMS 

CROSS-REFERENCE TO RELATED 5 
APPLICATIONS 

Not applicable. 

STATEMENT REGARDING FEDERALLY 
SPONSORED RESEARCH OR DEVELOPMENT 

Not applicable. 

BACKGROUND OF THE INVENTION 

The field of the invention is medical imaging methods and 15 
systems. More particularly, the invention relates to a stan- 
dard system which can be used to interface with any of 

several different imaging system types. 

Traditionally medical facilities have concentrated their 
efforts on providing the best possible medical services to 
patients. One area in which the quality of medical services 
has progressed extremely quickly is in the medical imaging 
disciplines or modalities which include radiography, 
fluoroscopy, angiography, magnetic resonance (MR) 
imaging, ultrasound, nuclear medicine (NM) and computer 
tomography (CT). 

While each of the separate imaging modalities can be 
used to generate medical images, the medical imaging 
industry has recognized that each modality is particularly 
suited for certain imaging techniques and that some modali- 
ties are better suited than others for observing specific 
anatomical phenomenon. For this reason, many medical 
facilities have acquired a plurality of imaging systems, each 
of which facilitates a different one of the imaging modalities. 
This is particularly true in large medical facilities which may 
have several imaging systems for performing each of the 
imaging modahties (i.e. several MR systems, several CT 
systems, etc.). Having several imaging systems each capable 
of facilitating a different one of the imaging modalities ^ 
enables a physician to select the best imaging modality for 
a particular imaging task and therefore increases the useful- 
ness of a resuhing image for achieving the particular task. In 
fact, many medical facilities compete for patients based on 
the abihties of their medical imaging departments. This 
imaging department competition places pressure on each 
medical facility to maintain state of the art imaging depart- 
ments. 

While imaging system quality has increased appreciably, 
the costs associated with maintaining a state of the art 
imaging department have also increased appreciably. 
Unfortunately, despite increased costs associated with pro- 
viding a state of the art imaging department, recently there 
has been mounting pressure on many medical facilities to 
reduce medical costs. For this reasons imaging departments 55 
are always looking for ways to decrease department costs 
while maintaining the highest possible service quality. 

In addition to imaging hardware and software costs which 
are substantial, another expensive component to any suc- 
cessful medical imaging department is imaging personnel 60 
which includes radiologists and technologists. A radiologist 
is a trained physician who specializes in radiology disci- 
plines and typically in other imaging modalities. Technolo- 
gists are supervised by the radiologists and perform most of 
the setup, imaging, filming and archiving of images. 55 

Basic training for a technologist in the imaging disciplines 
typically includes two years of on the job apprenticeship 
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which focuses on the human anatomy and physiology, 
imaging equipment theory and operation and imaging pro- 
cedures. In addition to basic training, many technologists 
obtain additional training in imaging specialties such as 
ultrasound, NM and radiation therapy. Training in each 
specialty typically takes about an additional year per spe- 
cialty. In addition, some technologists obtain further training 
in sub-specialties such as computer tomography CT, MR and 
angiography, each of which requires further on the job 
training. After a successfully completed apprenticeship a 
technologist performs procedures under the direction of a 
radiologist. 

Because of the differences in the imaging modalities, 
many medical facilities are staffed with a plurality of 
technologists, at least one technologist for each imaging 
modality practiced at the facihty. Thus, in some cases a 
facility may include at lease seven technologists, at least one 
technologist for each possible modality. While necessary to 
have a trained technologist in each imaging modality prac- 
ticed at a facility, such staffing requirements are extremely 
expensive. 

One solution to the excessive technologist stafifing prob- 
lem has been to train technologists in more than a single 
imaging modaHty. For example, one technologist may be 
trained in both NM and CT while another may be trained in 
fluoroscopy and radiography. 

Another solution to the excessive technologist staffing 
problem has been for medical facilities to share technolo- 
gists. Thus, for example, a CT technologist may divide her 
time between three separate medical facilities, every third 
day spent at a different one of the three facilities. 

While both of these solutions reduce costs associated with 
technologist staffing, each of the solutions is hampered by 
the current state of imaging systems and the way in which 
technologists are forced to interact with such systems and 
system information. An exemplary MR imaging session is 
instructive in understanding the difficulties associated with 
training a technologist in more than a single imaging modal- 
ity. 

Atypical MR imaging session comprises several different 
steps including scheduling, analyzing patient information, 
patient preparation and handhng, acquiring image data, 
displaying images, advanced processing of image data, 
filming display images, archiving display images, logging 
completed acquisitions and interpretation and reporting. 
Each of the different steps often requires a technologist to 
interact with one or several different department tools. For 
example, during scheduling a technologist typically uses a 
scheduling clipboard (i.e. paper on a clipboard) or the like to 
schedule imaging sessions during the coarse of a day. A 
scheduling table appears on the board which typically iden- 
tifies time, patient, the type of exam required (e.g. C-spine, 
brain, head, etc.), status, identification number, etc. 

As an alternative to a hardcopy clipboard, some facilities 
now have automated scheduling tools whereby a scheduling 
computer is used to generate a scheduling table which is 
consulted by the technologist throughout the course of a day 
to schedule and keep track of required imaging tasks. 

In addition to the scheduhng table, often a hardcopy (i.e. 
paper) requisition form will be provided for the technologist 
which includes additional patient identifying information 
(e.g. weight, height, sex, etc.), may list allergies, identifies 
the type of exam (e.g. c-spine, brain, head) to be performed, 
identifies the requesting physician and so on. Prior to 
imaging a requisition form is required to ensure that inad- 
vertent imaging is not performed on patients. Once again. 
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some automated facilities provide a computer for accessing 
requisition forms. 

After examining a requisition form and confirming autho- 
rization and prior to meeting a patient, the technologist 
typically confirms images which have to be acquired during 5 
a subsequent session. For example, while imaging a spinal 
section using the MR modality, it may be conventional to 
obtain a series of image slices along the length of the spinal 
section using a Tl FSE pulse sequence and a T2 FSE pulse 
sequence. In addition, a particular physician may routinely lo 
require a third series of image slices using an oblique axial 
FSE pulse sequence. To confirm required images technolo- 
gists often consult a required image guidebook which 
includes lists of required images and may include lists of 
images specially required by specific radiologists. In 15 
addition, a guidebook may also indicate required patient 
position and equipment position for each required image 
Moreover, each facility may also have a list of standard 
required images which must be consulted by the technolo- 
gist. 20 

During patient preparation and handling, the technologist 
greets the patient, explains the imaging procedure, helps the 
patient onto an imaging table and guides the patient into a 
first position required to collect a first of the required 
images. -^^ 

Next, during acquisition the technologist typically uses an 
acquisition computer to select specific images to be gener- 
ated by selecting image boundaries and image parameters. 
To this end the technologist often will acquire one or more 
localizing images which can be used to generally identify 
the position of anatomical structures within a patient's body. 
Viewing the localized image the physician selects required 
images to be generated and can select imaging parameters to 
use when generating the required images. 

Typically there are many different imaging parameters 
which can be selected and adjusted. In an effort to make 
acquisition computers easy to use, many acquisition com- 
puters provide an interactive interface including parameter 
icons for use with a pointing device to enable a technologist 
to select icons and change parameters with the click of a 
pointing device button (e.g. a mouse) or via a keyboard. To 
notify a technologist of all selectable parameters and iconic 
parameter selecting tools, interfaces of this type display 
virtually all parameters and associated icons on the interface 
for examination selection. 

In addition, to help a technologist select appropriate 
parameter settings the imaging guidebook may also include 
tables indicating standard parameter settings for each image 
to be formed . These standards may be selected by the facility 50 
generally and may also include specific required settings for 
each requesting radiologist. After selecting parameters the 
technologist causes the imaging system to acquire the 
selected images, the system storing the images electroni- 
cally. 55 

Next, the technologist may reposition the patient in a 
position optimal for collecting data for the second required 
image and thereafter follows the same procedure described 
above to collect required image data. 

After required images have been stored electronically the 60 
images are downloaded onto an archive system which 
typically includes a second computer in addition to the 
acquisition computer. By downloading digital images to a 
second computer the acquisition computer is freed to per- 
form a subsequent image acquisition. Many acquisition 65 
computers enable both archiving and acquisition at the same 
time so that, while image data corresponding to one patient 
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is downloaded to the second computer, image data corre- 
sponding to another patient can be collected. This simulta- 
neous dual function ability increases the throughput rate (i.e. 
patients/day) for the imaging system and thus overall effi- 
ciency. 

After the images are downloaded to the second computer, 
the technologist can use the second computer to analyze the 
images and perform advanced image processing. Thus, for 
example, where ten parallel and adjacent MR images have 
been collected which define a three dimensional data array, 
a technologist may want to generate a maximum intensity 
projection (MIP) using three of the ten images. To this end 
the technologist selects the three images to form the MIP and 
instructs the second computer to combine the three images 
to form the MIR Other advanced processes are possible and 
are contemplated. 

After advanced processing the technologist may select all 
or a subset of the original or advanced process images for 
generating film hardcopies for physician review. After 
selecting images for filming a filming machine, which is also 
controlled and maintained by the technologist, is used to 
generate required film pictures. After filming the pictures are 
provided to a physician for examination and thereafter are 
archived in a patient's file. In addition to filming, some 
systems also enable digital archiving so that digital images 
can be reaccessed using a computer or the like for review or 
for subsequent advanced processing. 

After a completed imaging session the technologist typi- 
cally uses a binded logbook notebook to document the 
completed session by indicating the date, time, patient name 
and number, examination type and so on. 

Thus, for a specific modality a technologist has to interact 
with several specific tools including a schedule, requisition 
forms, an image guidebook, an acquisition computer, a 
second advanced processing computer, a filming machine, 
an archiving database and a logbook. 

While learning to use these tools for a single modality is 
not terribly difficult, differences between similar tools used 
for different modalities complicates the process of becoming 
proficient in two or more modalities. Thus, for example, 
scheduling clipboards for one imaging system may be set up 
entirely differently than scheduling clipboards for another 
imaging system. Even where two imaging systems are both 
automated to include scheduling computers and even where 
each of the two automated systems are provided by the same 
vendor, each computer usually includes a different interface 
such that entry of scheduling information into the two 
computers is in a unique sequence and different information 
may be required for each system. 

Similarly, each system may have an entirely unique type 
of equisition form such that locating form information is a 
tedious task. This is true of both paper forms and automated 
computerized systems. 

Moreover, imaging guidebooks may be relatively com- 
plex and can become difficuU to use as radiologist's par- 
ticular requirements are added to the books. For example, at 
a large medical facility there may be more than 10 radiolo- 
gists who routinely require MR imaging. While each of the 
radiologists typically will require some identical images 
when a specific exam type structure (e.g. c-spine) is 
performed, many of the radiologists may require additional 
specific images which the particular radiologist finds helpful 
during diagnosis. In addition, while, for each required image 
there might be a typical patient position which is usually 
used to acquire the image, each radiologist may also have a 
slightly different preferred position which, in the radiolo- 
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gist*s judgment, yields a slightly better final image. In consistent with the specific modality and a pointing device 

addition, where radiologist's require specific system param- for moving a pointer icon about the display and for selecting 

eter settings the technologist also has to consult imaging displayed icons. When an icon is selected, the processor 

guidebook tables to determine required settings. Thus, a correlates the selected icon with a corresponding command 
complete imaging guidebook would have to indicate, for 5 and executes the command. 

each radiologist, required images for each body structure, xhe universal interface facihtates fast training of tech- 
where applicable, specific required patient positions for each nologists thereby enabling inexpensive cross-modality train- 
required image and required parameter settings for each jog and technologist substitution. To this end, it has been 

recognized that there are several basic system processes 

Clearly, using imaging guidebooks is tedious work. lO which are common to each of the known imaging modali- 

Unfortunately, the difficulty of using such guidebooks is ties. The universal interface is structured around the basic 

exacerbated by the fact that many of the guidebooks have processes to provide a feeling of comfort to a technologist 

unique forms, some systems include two or more guide- proficient in any imaging modality, 

books and many guidebooks can be supplemented as radi- it has also been recognized that, for each basic process, 
ologists are added to or removed from the facility staff. 15 ^^^^^ ^^^^^ ^ typical sub-process workflow which is 

Moreover, acquisition computer interfaces are often very routinely followed. Thus, preferably, each procedure which 
different and therefore, knowing how to use one interface is common to the first and second modalities includes 
does not render another interface intuitive. For example, the procedure specific sub-processes and the workspace 
acquisition computers for a MR system and a CT system includes a workflow navigation space in which, when a 
often have extremely different interfaces which require a function icon is selected, the processor displays a workflow 
technologist to step through very different protocols. This is icon set including a separate workflow icon corresponding to 
not surprising as the parameters which have to be selected each sub-process of the process associated with the selected 
for different modalities often are different. However, the function icon and for the specific modality. On each modal- 
tools provided for setting even similar parameters on two ity interface each of the navigation space and workflow 
different imaging systems often have a very different appear- space are similarly positioned and similarly color coded to 
ance and different operation. Thus, for example, to increase further render a technologist comfortable using each inter- 
a displayed parameter value one interface may require a user face. 

to place a cursor in a parameter box and type in a desired Moreover, the invention includes one or more data tables 

value. Another interface may include up and down arrow which are accessible by an acquisition computer for guiding 

icons adjacent a parameter box with a current parameter a technologist through the process of determining required 

setting displayed in the box, the setting increased by select- images and patient positions, setting proper imaging param- 

mg the up arrow via a pointing device. eters for each required image and selecting desired advanced 

Acquisition computer differences are exacerbated by the imaging processes, 
fact that most system interfaces are extremely cluttered as 

virtually every possible tool for setting acquisition param- BRIEF DESCRIPTION OF THE DRAWINGS 

eters is usually provided on the interface screen, this despite ^ . , , , ^ **y,x 

the fact that many tools are only rarely used by the tech- ^\^' ^ ^^§5^°^ ^y^^^"^ ^^^^^ 

nologist. While designed to help a technologist by indicating ^"^P^^^" mvention; 

all possible tools it has been recognized that such a crowded P^^- 2 is a schematic illustrating an exemplary data table 

screen actually reduces technologist efliciency as the loca- exemplary position image table according to present 

tions of the most widely used tools are obfuscated. invention; 

Furthermore, the interfaces and operation of the advanced FIG. 3 is a schematic illustrating an exemplary schedule 

processing computers, filming machines and archiving com- interface screen according to the present invention; 
puters for different processes (i.e. modalities) are also often 45 FIG, 4 is a schematic illustrating an exemplary patient 

very different. information interface screen; 

All of the interfacing problems described above are FIG. 5 is a schematic illustrating an exemplary acquire 

exacerbated where a single technologist works between interface screen; 

facilities as facility unique interfaces, guidebooks and piG. 6 is a schematic illustrating an exemplary image 
schedules have to be decrypted by visiting technologists 50 viewing display screen; 

prior to efficient system use. ^ ^ schematic illustrating an exemplary analyze 

SUMMARY OF THE INVENTION interface screen; 

An exemplary embodiment of the invention includes a .^^^ ^.schematic illustrating an exemplary utilities/ 
universal interface usable with at least first and second 55 P'*^^^^^^ interlace screen; 

different imaging modalities, each modality including func- P^^. 9 is similar to FIG. 8, albeit iflustrating a second 

tions or procedures which are common to each of the first ^^^^^ ^he protocol interface screen; and 

and second modalities, each separate instance of the inter- FIG. 10 is similar to FIG. 9 albeit illustrating yet another 

face used with only a specific one of the first or second level of the protocol interface screen. 

modalities. Tlie interface comprises a display, a programmed 60 

data processor for providing a uniform interface image on DESCRIPTION OF THE PREFERRED 

the display despite the specific modality, the uniform inter- EMBODIMENT 
face image comprising a function navigation space including 
function icons corresponding to procedures which are com- 
mon to both the first and second imaging modahties and a 65 While the present invention is meant to be used with any 

workspace adjacent the function navigation space for imaging system despite the modality facilitated by the 

displaying, analyzing and manipulating images of a type system, in order to simplify this explanation the invention 



I. HARDWARE 
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will be described in the context of an exemplary MR system. which move the patient cradle and transport the patient to the 

To this end, referring first to FIG. 1, there is shown the major desired position for the scan. 

components of a preferred MR system which incorporates The gradient waveforms produced by the pulse generator 

the present invention. The operation of the system is con- module 121 are appHed to a gradient amplifier system 127 

trolled from a universal interface 100 which includes an 5 comprised of G^, G^ and G^, amphfiers 136, 137 and 138, 

interface processor 101 that scans a keyboard 102 and respectively. Each amplifier 136, 137 and 138 is utihzed to 

receives inputs from a human operator through a display excite a corresponding gradient coil in an assembly gener- 

screen 104. Screen 104 may be either a plasma/touch screen designated 139. The gradient coil assembly 139 forms 

or a more conventional display on which a pointer cursor is of a magnet assembly 141 which includes a polarizing 

provided which can be moved via mouse 103. For the ^^^^^^ ^^^^ produces either a 0.5 or a 1.5 Tesla 

purposes of the present invention it will be assumed that polarizing field that extends horizontally through a bore 142. 

mouse 103 is used to move a pointing icon or cursor on Operation of coils 139 is well known in the art and therefore 

display 104. is not explained here in detail. 

Interface processor 101 commumcates through a commu- Located within the bore 142 is a circular cylindrical 

nications link 116 with an applications interface module 1 17 whole-body RF coil 152, This coil 152 produces a circularly 

in a separate computer system 107. Through the keyboard polarized RF field in response to RF pulses provided by a 

102 and mouse 103, a technologist controls the production transceiver module 150 in the system control cabinet 122. 

and display of images by a processor 106 in the computer j^^^^ p^i^^s are amplified by an RF amplifier 151 and 

system 107, which connects to display 104 through a video coupled to the RF coil 152 by a transmit/receive switch 154 

cable 105 and processor 101. 20 which forms an integral part of the RF coil assembly. 

Computer system 107 includes a number of modules Waveforms and control signals are provided by the pulse 

which communicate with each other through a backplane. In generator module 121 and utilized by the transceiver module 

addition to application interface 117 and processor 106, 150 for RF carrier modulation and mode control. The 

these include a CPU module 108 which controls the back- resulting NMR signals radiated by the excited nuclei in the 

plane and which connects the computer system 107 through 25 patient may be sensed by the same RF coil 152 and coupled 

a bus 110 to a set of peripheral devices, including disk through the transmit/receive switch 154 to a preamplifier 

storage 111 and tape drive 112. CPU module 108 is also 153, jhe amplified NMR signals are demodulated, filtered, 

linked to a film machine 90 via a bus 92. The computer ^nd digitized in the receiver section of the transceiver 150. 

system 107 also includes a memory module 113, known in xhe transmit/receive switch 154 is controlled by a signal 

the art as a frame buffer for storing image data arrays, and 30 from the pulse generator module 121 to electrically connect 

a serial interface module 114 that links the computer system the RF amplifier 151 to the coil 152 during the transmit 

107 through a high speed serial link 115 to a system interface j^Q^e and to connect the preamplifier 153 during the receive 

module 120 located in a separate system control cabinet 122, mode. The transmit/receive switch 154 also enables a sepa- 

The system control 122 includes a series of modules rate RF coil (for example, a head coil or surface coil) to be 

which are connected together by a common backplane 118. 35 used in either the transmit or receive mode. 

The backplane 118 is comprised of a number of bus The NMR signals picked up by the RF coil 152 are 

structures, including a bus structure which is controlled by digitized by the transceiver module 150 and transferred to a 

a CPU module 119. The serial interface module 120 con- memory module 160 which is also part of the system control 

nects this backplane 118 to the high speed serial link 115, 122. When the scan is completed and an entire array of data 

and pulse generator module 121 connects the baclq)lane 118 40 has been acquired in the memory modules 160, an array 

to interface 100 through a serial link 125. It is through this processor 161 operates to Fourier transform the data into an 

link 125 that the system control 122 receives commands array of image data. This image data is conveyed through the 

from the technologist which indicate the scan sequence that serial link 115 to the computer system 107 where it is stored 

is to be performed. memory 113. In response to commands received from 

The pulse generator module 121 operates the system 45 interface 100, this image data may be archived on the tape 
components to carry out the desired scan sequence. Module drive 112 or memory 111, or it may be further processed by 
121 produces data which indicates the timing, strength and the image processor 106 and conveyed to interface 100 and 
shape of the RF pulses which are to be produced, and the presented on display 104. 
timing of and length of the data acquisition window. Module TNrTCDi7Ar't? 
121 also connects through serial link 126 to a set of gradient 50 INTERbACb 
amplifiers 127 and conveys data thereto which indicates the All of the hardware described above is conventional. The 
timing and shape of the gradient pulses that are to be present invention resides within interface 100 and system 
produced during the scan. The pulse generator module 121 107 and is manifest in two general concepts, a universal 
also receives patient data through a serial link 128 from a interface and modahty guidance tools which determine 
physiological acquisition controller 129. The physiological 55 information which should be provided to a technologist via 
acquisition control 129 can receive a signal from a number the universal interface during an imaging session, 
of different sensors connected to the patient. For example, it With respect to the universal interface, processes which 
may receive ECG signals from electrodes or respiratory are common to virtually all imaging modahties have been 
signals from a bellows and produces pulses for the pulse identified, sub-processes which are common to each process 
generator module 121 that synchronizes the scan with the 60 in a specific modality have been identified and parameters 
patient's cardiac cycle or respiratory cycle. And finally, the which are common to each sub-process in a specific modal- 
pulse generator module 121 connects through a serial Hnk ity have been identified. Based on the common processes, 
132 to scan room interface circuit 133 which receives sub-processes and parameters a set of interface screens have 
signals at inputs 135 from various sensors associated with been developed which, despite use with each of the 
the position and condition of the patient and the magnet 65 modalities, have many identical characteristics which are 
system. It is also through the scan room interface circuit 133 recognizable and foster a comfortable feeling, even for a 
that a patient positioning system 134 receives commands technologist using a system for the first time. 
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With respect to the modality guidance tools, it has been Referring still to FIG. 2, for each required image in third 

recognized that each hospital, and perhaps each radiologist column 208, a plurality of tools and default parameter 

at a hospital, may require different images to be generated settings for a specific radiologist and specific exam type are 

during a specific type of exam, may require specific patient listed in fifth column 214. Importantly the parameters listed 

positions during generation of data for certain images and 5 in fifth column 214 are only the most important parameters 

may also require specific parameter settings for each image and the most often modified parameters for a particular 

generated. The modality guidance tools consist of a set of imaging modahty. The tools Hsted in column 214 are the 

tables stored in computer memory which are used to auto- tools which are necessary for modifying the default param- 

matically determine required settings and images and for eter values, 

indicating suitable patient position when a specific radiolo- It should be appreciated that while the tables shown in 

gist orders an exam of a specific type. After determining FIG. 2 provide data and position images for each radiologist 

required images, positions and parameters using the guid- for MR imaging, a separate yet similar table is provided for 

ance tools, icons are provided via the universal interface to each of the other six imaging modalities in imaging system 

guide the technologist through a properly orchestrated imag- memories earmarked specifically for those other modalities, 

ing protocol. Pqj- example, a CT system would include tables similar to 

Referring now to FIG. 2, therein are shown two exem- table 200 and 202 which identify each radiologist, exam 

plary tables stored in memory 113 which are accessible to types, required images, position image specifiers and param- 

processor 106 via the common backplane and which are ^^^^^ tool specifiers 

used by processor 106 to provide interface information to n r - ♦ nrr^ 1 1 ■ * r 

/ ^ 1A1 f -J- ♦ u 1 • * ♦! ^ Referrmg now to rIG. 3, an exemplary interface screen 

console processor 101 for guidmg a technologist through an .... -,1 /. 1 -./i/- -n ..10 

imaging process. The two ttbles include a dati table 2(1 and 20 216 which is provided on display 104 is illustrated. Screen 

a position image table 202. Exemplary data table 200 216 generally includes two spaces including a domain space 

includes five separate types of information. The first type of ^18 and a dynamic space 220. Generally, within domain 

information includes radiologist indicators which, for the space 218, three separate domain icons including a schedule 

purposes of this explanation are shown as the names of icon 220, a log book icon 224 and a utility icon 226, are 

different radiologists in a first table column 204. While two 25 provided. Each of the domain icons 222, 224 and 226 can be 

radiologist names. Dr. Welby and Dr. Hanson are shown, it selected by using mouse 103 (see FIG. 1) to move a pointing 

is contemplated that there may be many more radiologists cursor or icon 228 about screen 216. Each of domain icons 

listed. 222, 224 and 226 is used to open an entirely different 

A second type of information includes exam types which application of the inventive interface system. Generally, 

are shown in a second column 206. For each doctor listed in 30 schedule icon 222 enables a technologist to review and 

first column 204, every possible type of exam is listed in modify a patient schedule corresponding to a specific imag- 

second column 206. Exemplary exam types shown in FIG. ing system. Log book 224 enables a technologist or radi- 

2 include a C-spine type and a brain type. ologist to review a log book of completed imaging sessions 

For each exam type listed in the second column 206, a ^"d, where images are digitally archived, enables the tech- 
plurality of required images are listed in a third column 208. 35 "olog^st to retrieve and review the archived images. Utilities 
For example, for Dr. Welby's C-spine exam type, required icon 226 allow the technologist to perform a number of tasks 
images include a saginal T2 FSE sequence, a saginal Tl FSE including, but not limited to, retrieving archived records and 
sequence and an oblique axial FSE sequence. Although not modifying the default settings which are illustrated in the 
illustrated, a similar list of required images would be pro- tables of FIG. 2. 

vided for Dr. Welby's brain type examination and for each 40 Initially, when a technologist logs on to interface 100, the 

of the other types of examinations Dr. Welby often requests. technologist is provided with a simple screen including 

In addition, a similar list of required images would be domain space 218, dynamic space 220 and the domain icons 

provided for each exam type Dr. Hanson routinely requests 222, 224 and 226 spaced in a single row within domain 

and so on. space 218. Spaces 218 and 220 are differently colored to 

For each required image in third column 208 a position 45 easily distinguish there boundaries, 

image address corresponding to an image stored in position Referring still to FIG. 3, when schedule icon 222 is 

image table 202 is provided. For example, for the saginal T2 selected processor 106 accesses a current patient schedule 

FSE required image for Dr. Welby's C-spine exam type, the which is stored in memory 113 and performs four separate 

position image address in fourth column 210 is 001. Address functions via control processor 101 which are observable on 

001 points to an image 212 in position image table 202 50 screen 216. First, processor 106 indicates selection of the 

which is stored at address 001. Image 212 shows a patient schedule icon 222 by providing a "digital tab" 230 which 

laying flat on her back with legs extended on an imaging encompasses the schedule icon 222 within dynamic space 

table. It should be noted that while image 212 and other 220 to distinguish selected icon 222 from the non-selected 

images in position image table 202 are shown as simplified icons 224 and 226 which remain within domain space 218. 

silhouette drawings, more detailed images are contemplated 55 Second, processor 106 displays the current patient schedule 

and in fact are preferred with the present invention. The on a digital clipboard metaphor image 232. Metaphor image 

more detailed images may include actual digital pictures of 232 resembles a conventional hardcopy clipboard which is 

a model positioned in precisely required positions for imag- typically used by technologists to maintain a patient sched- 

ing purposes. In addition, the position images may also ule. The schedule on image 232 may include any of several 

include text descriptions of ideal patient positions to help a 60 of different types of information, the exemplary schedule of 

technologist identify an ideal position. Moreover, while a FIG. 3 including a time column 234, a patient name column 

single position image table is illustrated in FIG. 2, memory 236, a patient ID column 238, an exam type column 240 and 

113 (see FIG. 1) may include a separate position image table a status column 242 which indicates whether or not an 

for each radiologist listed in data table 200. Furthermore, the imaging session has been completed, is in a waiting state or 

position images in table 202 may also indicate desired 65 is scheduled for a subsequent time, 

equipment positions relative to a patient (e.g. location of a Third, processor 106 provides display modifying icons 

CT camera relative a patient). including an add patient icon 244 and an update patient icon 
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246 which can be used to modify the schedule on image 232. 
Although not illustrated, it is contemplated that when one of 
the modify icons 244 or 246 is selected, a window opens up 
which provides tools to enable a technologist to add, delete 
or modify information corresponding to any of the informa- 
tion listed on image 232. Methods and software for per- 
forming these tasks are well known in the art and therefore 
are not explained here in detail. 

Fourth, processor 106 provides a search box 248 and a 
find icon 250 which can be used together to locate a specific 
patient on patient schedule image 232. To this end, a 
patient's name can be provided in box 248 and thereafter 
icon 250 can be selected at which time processor 106 
searches the table stored in memory 113 and locates the 
indicated patient. 

In addition to the four identified processor functions 
which are recognizable via display 104, processor 106 also 
provides a memory 113 address for each patient exam 
provided on image 232, the memory address invisible to the 
technologist but nevertheless linked to a specific exam. A 
specific exam and associated memory address operate like 
an Internet hyperlink to allow a technologist to access a 
radiology request form associated with the specific exam 
and stored at the memory address by selecting the specific 
exam using icon 228. 

Referring to FIGS. 1 and 3, using mouse 103 a technolo- 
gist can select any of the icons illustrated on screen 216 by 
placing icon 228 on the illustrated icon and pressing a 
recognizable sequence of mouse buttons. While schedule 
image 232 is being examined, a technologist can place 
pointing cursor 228 on any patient name which appears on 
image 232 and, by selecting the patient's name, can access 
a digital radiology request form corresponding to the patient 
and the specific exam. In particular, prior to every imaging 
session, a technologist must review a radiology request form 
to make sure that the imaging session has been authorized by 
one of the radiologists and to identify any special instruc- 
tions provided by the radiologist. In FIG. 3, it is assumed 
that for each exam having a "complete" status, the tech- 
nologist has already reviewed a radiology request form, met 
with the patient, set up an exam, performed an imaging 
session to generate imaging data and has downloaded the 
imaging data to either of devices 111 or 112. However, for 
the exams having a "waiting" status indicator in column 242 
of image 232, the technologist has not yet accessed a 
radiology request form. 

To access the radiology request form, as indicated above, 
the technologist uses cursor 228 to select the next exam 
having a "waiting" status in image 232. In this case, the 
technologist selected Mike Jones. When selected, the entire 
row of data in image 232 corresponding to Mike Jones is 
highlighted. A second selection causes processor 106 to 
move from a scheduling domain into an imaging session 
domain. 

Referring now to FIG. 4, an interface screen 250 provided 
by processor 106 when Mike Jones is selected via image 232 
is illustrated. Screen 250 includes both the domain space 218 
and the dynamic space 220. However dynamic space 220 is 
now divided into a primary navigation space 252 which, in 
the exemplary illustration, includes a single column along 
the left-hand side of dynamic space 220 and a work space 
254. 

Processor 106 provides a plurality of function icons in 
navigation space 252. Importantly, the function icons in 
space 252 have been selected to mirror processor functions 
which are common to virtually all imaging modalities. 
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Therefore, the navigation space icons can be provided on 
separate interfaces corresponding to every type of imaging 
system. Thus, for example, while the interface screen 250 
shown in FIG. 4 is specifically for an MR system, an 

5 interface having both the domain space 218, including icons, 
and the navigation space 252, including navigation space 
icons, would be provided for a CT interface, a radiography 
interface, a fluoroscopy interface and so on. This feature 
which clearly defines specific and common functions 

10 between all imaging systems fosters a sense of comfort 
when a technologist moves from one imaging system type to 
another imaging system type or from one medical facility to 
another. This common interface also helps guide the tech- 
nologist through an imaging session. 

15 Referring to FIG. 4, the navigation space icons includes a 
patient information icon 256, an acquire icon 258, a view/ 
film icon 260, an analyze icon 262 and a close icon 264. It 
should also be noted that all of the navigation space icons are 
arranged in a single column which follows a normal work- 

20 flow pattern from the patient information icon 256 through 
the acquire icon 258, the view/film icon 260, the analyze 
icon 262 and the close icon 264. This feature enables a 
technologist to clearly see a required or desired workflow 
pattern. 

Referring to FIGS. 1, 3 and 4, when a technologist selects 
Mike Jones on interface 262 processor 106 accesses memory 
113 and retrieves the radiology request form which is stored 
at the address corresponding to the exam for Mike Jones, 
Retrieving the radiology request form processor 106 dis- 
plays the request form in workspace 254. In addition, to 
clearly indicate that the information in workspace 254 is 
associated with the patient information icon 256, processor 
106 encompasses patient information icon 256 in workspace 
254 by providing a digital tab 266 which includes patient 
information icon 265 in workspace 254. 

Referring still to FIG. 4, an exemplary radiology request 
form 255 includes a requisition number 268, patient iden- 
tification information (e.g. name, id number, age, weight, 
etc.) generally referenced by numeral 270 and exam request 
information indicating the type of exam required, relative 
patient history, a required protocol, the referring doctor, the 
responsible radiologist and so on, collectively referred to by 
numeral 272. In addition, processor 106 may include a space 
274 where the technologist can input her name to complete 
the radiology request form. Although not illustrated, iconic 
tools for modifying the protocol or other request form 
information may be provided within workspace 254, the 
tools selectable via pointer icon 228. 

50 After the technologist has examined the radiology request 
form to determine a required protocol and ensure proper 
authorization of the exam, the technologist selects acquire 
icon 258 via icon 228. Referring now to FIG. 5, when 
acquire icon 258 is selected, processor 106 indicates selec- 

55 tion of acquire icon 258 by providing a digital tab 276 which 
encompasses acquire icon 258 within workspace 254. In 
addition, processor 106 divides workspace 254 into a work- 
flow navigation space 278, an imaging window 280 and a 
parameter settings space 282. Workflow navigation space 

60 278 preferably includes a single column adjacent navigation 
space 252. In the exemplary illustration, the imaging win- 
dow is in the upper right-hand comer of workspace 254 
while parameter settings space 282 encompasses the remain- 
der of workspace 254. 

65 Referring now to FIGS. 1, 2, 4 and 5, in addition to 
dividing workspace 254 as described above, when acquire 
icon 258 is selected, processor 106 identifies the radiologist 
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indicated on the request form 255 and the required protocol correct position for the patient and can review all of the 

from the request form and uses those two pieces of infor- default settings for the exam type for the particular radiolo- 

mation to access a specific set of required images and gist. If the radiologist requested specific parameter values 

parameters in table 200 which is stored in memory 113. In which are different than the default settings, the technologist 

the present example, processor 106 identifies Dr. Welby and 5 can use the iconic tools to modify the parameter values, 

his C-spine protocol as the exam type and accesses memory Qnce the patient is properly positioned and all of the 

113 to identify four required images including a "localizer" parameter values are properly set, the technologist selects an 

a "Sag p FSE", a "Sag Tl FSE" and a "ob ax FSE" image. ^^^^ ^^^^ indicating to the imaging system that a 

For each of these required images^^processor 106 provides a j^^^j.^^^ ^ ^^^^^^ ^e generated, 

separate icon within space 278. The icons are identified by _ ^ r 1 i- • • , • 1 

numeral 284, 286, 288 and 290, respectively and are col- Thereafter, a localizer image is generated via the system 

lectively referred to as a workflow icon set. Each of the of FIG. 1 and the image appears m window 280 which is 

workflow icons includes a virtual button and an LED observed by the technologist. If desired, the technologist can 

illustration which, when the icon is selected, lights up. For generate several different localizer images until patient and 

example, localizer icon 284 includes LED illustration 292. anatomical body position is well known. 

Importantly, just as the function icons in navigation space Next, referring to FIGS. 2 and 5, after a suitable localizer 

252 are provided in a typical workflow form from top to image has been generated, the technologist selects icon 286 

bottom, workflow icons in space 278 are also provided in a to set up the Sag T2 FSE imaging session and then generates 

similar top to bottom workflow pattern. To this end, localizer the required image set. Although not illustrated, when icon 

icon 284 is first listed and therefore generally first selected 286 is selected, processor 106 provides space 282 in a 

to generate a "localizer" image in image window 280 which 20 manner similar to that illustrated in FIG. 5, albeit providing 

can be used to determine general patient and anatomical parameter values, tools and a position image window cor- 

body position with respect to system hardware. TTiereafter responding to the data stored in table 200 which is associated 

other icons 286, 288 and 290 are selected o generate three 5 ^ .^^^ . ^^^^ 

different types of image sots and the typical order is Sag Tl ^^^^^^^ ^ ^^^^ .^^ 

FSE, Sag T2 FSE and ob ax FSE. 95 ^lo/^ . u 1 • ^ ^- * j- 1 

r, r • . r^T^o u i ^' spacc 282, the technologist positjons the paticnt accordmgly 

Refernng to FIGS. 1, 2 and 5, initiafly, when localizer *^ , /^r ^^p„„u «„,„™=t««, 

-lo^ • 1 * J • • J J • J -lOA and may modify some of the default parameters pnor to 

icon 284 IS selected, no image is provided in window 280. i ^- ^ • '^^i ^ / • r • 

However, all of the parameters and tools indicated in the ^^^^^^^"g ^^e accept icon 312 to generate a senes of images 

fifth column 214 of table 200 which correspond to the corresponding to the desired protocol, 

localizer image for Dr. Welby's C-spine type exam are 30 ^ ^^^"iilar process is performed to generate a series of 

provided in space 282. All of the parameter values provided '^^^^^ each of the Sag Tl FSE and Ob Ax FSE icons 

are the default values indicated in table 200. Each of the 288,290, respectively. All of the generated images are stored 

parameter values can be changed using iconic tools provided ^° memory 113. 

in space 282. For example, to change the pulse sequence of Referring now to FIG. 6, after all of the data or all of the 
the plane through which a localizer image is taken, a 35 required images has been collected and stored in memory 
technologist can select a technique tool icon 300 via icon 113, the technologist selects the view/film icon 260 to 
228. Although not illustrated, when one of the tool icons review the collected images. When icon 260 is selected, 
such as technique icon 300 is selected, it is contemplated processor 106 indicates selection of icon 260 by providing 
that a window opens up providing tools to modify all of the a digital tab 314 which encompasses icon 260 in workspace 
parameters corresponding to technique (e.g. pulse sequence 40 254. In addition, processor 106 maintains workflow icons 
and plane). Similarly, a range tool icon 302 is provided, a 284, 286, 288 and 290 in space 278. Processor 106 also 
scan tool icon 304 and a processor tool icon 306 each of provides film and display tool icons 316 and 318, 
which, when selected, opens a window for modifying respectively, behind which various tools are hidden in win- 
parameters associated therewith, dow form, the tools descried in more detail below. 

Referring still to FIGS. 2 and 5, the parameter values and 45 The technologist can select any of the workflow icons 

tools in space 282 are referred to generally as a commonly 284, 286, 288 or 290 to display corresponding images in 

modified parameter set and associated tools. In addition to space 254. As illustrated, when icon 286 is selected, an 

displaying all of the parameters values and tools indicated in image in a first box 320 includes an axial slice through a 

column 214 of table 200 in space 282, processor 106 also patient and provides phantom lines collectively referred to 

provides a tool box icon 308 which, when selected, opens 50 by numeral 322, each of the lines 322 corresponding to a 

another window which includes a relatively large number of separate one of the following images displayed in space 254. 

additional parameter values and tools which are only sel- Processor 106 divides space 254 equally into a plurality of 

domly used during localizer imaging. These additional val- separate spaces and provides a separate image in each one of 

ues and tools are referred to generally as a seldom ly modi- the spaces, each image corresponding to a separate one of 

fied parameter set and associated tools. The idea here is that 55 lines 322. 

by providing seldomly used tools in a hidden yet easily After reviewing images, a technologist can use interface 

accessible format (i.e. "behind" a tool box icon), the screen 100 to perform advanced processing techniques on the 

display is less cluttered and routinely used tools and selected images thereby generating even more informative images 

parameters can easily be located. Preferably processor 106 for a radiologist to use. To this end, referring to FIG. 7, a 

also provides an image position window 310 in space 282 60 technologist selects analyze icon 262, Referring also to FIG. 

which indicates a patient position via an image from the 1, when icon 262 is selected, processor 106 indicates selec- 

position image table 202 which is indicated in fourth column tion of icon 262 by providing a digital tab 324 which extends 

210 of data table 200. In the present example the displayed from workspace 254 and encompasses icon 262. In the 

image in window 310 is the first image in position image interest of simplifying this explanation, while each separate 

table 202 as indicated in column 210. 65 radiologist at a facility may have a specific set of advanced 

Thus, after selecting localizer icon 284 the technologist imaging processes that the radiologist routinely like to have 

can observe the image in window 310 to determine the performed, it will be assumed that in this case the protocol 
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for advanced image processing for each exam type is example, one of the tools may be a measuring tool which 

specified by the facility and is not radiologist dependent. allows a technologist to select two points on an image and 

Thus, for a C-spine exam type, the advanced imaging measure the distance therebetween. Tools of this nature are 

protocol is identical for every radiologist at the facility. well known in the art and therefore are not described here in 

While this assumption is made in the interest of simplifying 5 detail. 

this explanation it should be recognized that a table similar addition, with images displayed, the technologist may 

to the table illustrated in FIG. 2 could be provided in select the fihn tools icon 316 which opens a film tools 

memory U3 (see FIG. 1) which would be accessed by window 350 as illustrated. Window 350 includes filming 

processor 106 each time the analyze icon 262 is selected. In tools including a print sheet icon 352, a pnnt senes icon 354 

this case, the processor 106 would identify radiologist and lo a° "^^S® ^^^^ ^ ^aore icon 358. When prmt 

exam type and access an advanced imaging table to deter- ^heet icon 352 is selected, processor 106 provides data 

mine a required and radiologist/exam type specific advanced corresponding to every image m space 252 to film machine 

imaging protocol. Thereafter, the processor 106 would pro- ^® ^^^^es film machine 90 to generate either one or a 

vide a workflow icon for each step in the advanced imaging plurahty of hardcopy film images of the displayed imaged, 

protocol in a workflow navigation space. 15 When print series icon 354 is selected, processor 106 

T ,u * 1 n u ^ *u * p downloads data corresponding to all images corresponding 

In the present example it will be assumed that tor a ^ ,„ : , 7^ m 

c-spine examination at Dr. Welby's medical facility the to all of the workflow irons .n space 278 to fito 

-i , 1 r. ,1 J- 1 • * ' 1 J rL \ thereby causing machine 90 to generate hardcopy film 

standard protocol for all radiologists includes five steps . r i_ j- • a- « c . 

which include, in the order of normal workflow, building a '^Sf'^ °f «!*'^ corr^pondmg image. To use icon 356. first 

three-dimensional data model, isolating a right artery, rotet- 20 a technolog|st would «aect one or a plurality of irn^es 

in and filmin the ri ht arter isolaSn a left arte and displayed within space 254 and thereafter select icon 356 to 

ing an "^^^S . ^ [^^ f f ^ ^ a ery an QXdiSJt the selected image from space 254. It so contemplated 

rotating and filming the left artery. To this end, processor 106 ♦u , *u «i ■ * f u u ju i *u 

•J, a ^ a • • ^ a ' luat Other nlming tools could be accessed by selecting the 

provides five workflow icons in workflow navigation space • -^^o t> 

JI-o . • 1 J- u -u J 1 • i'>^ • 1 * more icon 358, 

278, the icons including a build model icon 326, an isolate ^ r ■ • r ^^ • ^ 
right icon 328. a cine/film right icon 330, an isolate left icon 25 . Referring again to FIG. 3, after filming is complete the 
332 and a cine/film left icon 334. imagmg process is essentially completed for the patient and 
„ ^ . .„ « . . , , , , . the process would begin again with the next patient on 
Referring still to FIG. 7, when the build model icon 326 c^h.H„i. ^^.o^ oio 
lAiC 4u-^« * 1 : iio scneauie image zjz. 



is selected, processor 106 provides three tool icons 336, 338 
and 340 in work space 254. Referring also to FIG. 6, icon 



Referring again to FIGS. 1 and 3, when a technologist 



J * 11 f *!. • • *u ' * 30 selects logbook icon 224, ahhough not illustrated, processor 

336 corresponds to all of the images m the image set • * i -i-f^i i j- i. i 

• * J -^u *u o T-oT- 7 1 o- -1 1 • 106 indicates selection of icon 224 by providing a digital tab 

associated with the Sag T2 FSE protocol. Similarly, icons ... _ , c -im j - -.-»>i r 

no J i/in ^4 ^^ c.u • -/j -^u which exlcnds from space 220 and encompasses icon 224. In 

338 and 340 correspond to all of the images associated with , . . i l i . t_ • 

the Sag Tl FSE protocol and the Ob Ax FSE protocol, add'l.on. proc^r 106 generates a logbook me^ 

respectively. The technologist selects one of icons 336. 338 lUustrated) wtach m similar to the patient s schedule 

^ ^ 35 metaphor image 232, the logbook metaphor image provided 



or 340 to identify images which will be used to build the 
three-dimensional model and thereafter selects a build icon 



in dynamic space 220. To distinguish the logbook metaphor 



342 instructing 106 to use the selected image set to buUd a ^''S^ ^" ^^"^^ ^32. preferably the logbook 

three-dimensional model. ^If* » ^'Shtly different appei^ance. For example, the 

logbook metaphor image may mclude a spiral binder appear- 
After the three-dimensional model is generated and stored ance along the lefthand edge of the image. Information 
m memory 113 the technologist steps through each of the ^rovid^ on the logbook image is similar to the information 
isolate nght icon 328, cine/film right icon 330, isolate left provided on patient schedule image 232 except that the 
icon 332 and cine/fikn left icon 334 to select the specific bgbook information includes information corresponding to 
portions of the three-dimensional model for imagmg and to completed imaging sessions. To this end, exemplary log- 
generate images of the selected porbons. Isolating, rotatmg 45 b^i, ij^age information may include a date, a patient's 
and filming are processes which are well known in the art ^ame, the patient's ID number, an exam type (e.g. brain, 



and tools therefore are also well known in the art and 



head, C-spine, etc.) and a column for indicating whether or 



therefore those processes and tools are not explained here in ^ot exam images have been archived, 
detail. Itshould besufficienttostate that all of the tools and ^^^^^^ pj^, s, when a technologist selects 

an image display ^^ndow for performmg each of those ^^^-^^^ .^^^ ^26, processor 106 indicates selection by 

processes would preferably be provided withm space 254. providing a digital tab 348 which extends from space 254 

Referrmg once again to FIG. 6, after performing advanced into domain space 218 and encompasses utilities icon 226. 

image processing on the images the technologist selects the Utilities applications associated with icon 226 may include 

view/film icon 260 once again to display the original images a number of different functions, two exemplary functions 
or the advanced processed images within space 254. Where 55 including an archive function and a protocols function which 

advanced processed images have been generated, another are used to store or retrieve images and to modify data and 

workflow icon (not illustrated) would be provided below image tables (see FIG. 2), respectively. To this end, when 

icon 290 which could be selected by the technologist to view utilities icon 226 is selected, processor 106 again separates 

the advanced process images. dynamic space 220 into a navigation space 252 and a work 

Having selected the view/film icon 260, processor 106 60 space 254. Processor 106 provides both an archive icon 350 
again enables the technologist to select any of the workflow and a protocols icon 352 in space 252, Each of icons 350 and 
icons 284, 286, 288 or 290 to view corresponding images. In 352 selectable using icon 228. When archive icon 350 is 
FIG. 6 icon 286 has been selected and therefore correspond- selected, processor 106 provides iconic tools within work- 
ing images are illustrated. With the images illustrated, the space 254 which enable a technologist to retrieve and view 
technologist can select display tools icon 318 which opens 65 archived images corresponding to previously performed 
a window (not illustrated) including a plurality of other tools imaging sessions. Tools for this purpose are known gener- 
that allow a technologist to examine displayed images. For ally in the art and therefore are not described here in detail. 
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It is contemplated that a medical facility would likely 
have a set of internal pohcies concerning which personnel 
are authorized to modify default exam protocols and param- 
eter values in data tables like table 200 illustrated in FIG. 2 
for semi-permanent storage in memory 113. For example, 
one broad rule might be that only a radiologist is authorized 
to modify her default protocols and parameter values. To 
enforce such a rule a password security feature may be 
provided when protocols icon 352 is selected, processor 106 
only allowing protocol modification by an authorized radi- 
ologist. While a secure system is preferred, it will be 
assumed that a technologist can modify default protocols 
and parameter values in table 200 to simplify this explana- 
tion. 

When a technologist selects icon 352, processor 106 
provides a digital tab 354 which extends from workspace 
254 into space 252 and encompasses icon 352. In addition, 
processor 106 provides a protocol table 356 which corre- 
sponds to data table 200 in FIG. 2. The exemplary protocol 
table 356 includes two columns, first column 358 listing 
facility radiologists identified in table 200 and a second 
column 360 listing exam types (e.g. C-spine, brain, etc.) 
routinely requested by a corresponding radiologist. To 
modify the default settings within table 200, the technologist 
uses icon 228 to select a particular radiologist and exam 
type. For example, a technologist may wish to modify the 
default settings for Dr. Welby's C-spine exam type. To this 
end, the technologist selects Dr. Welby's C-spine from table 
356. 

Referring now to FIG. 9, when Dr. Welby's C-spine exam 
is selected, processor 106 provides a modify images icon 
362, a modify preferences icon 364 and an accept icon 366 
in workflow navigation space 278. In addition, processor 
106 provides two tables, a first table 368 indicating addi- 
tional image types which maybe required and a second table 
370 indicating already selected required image types for Dr. 
Welby's C-spine exam. Add and delete icons 372 and 374 
are also provided. By selecting an image type in table 368 
and then selecting the add icon 372, the selected image type 
in table 368 is moved into table 370 and processor 106 
updates table 200 to add the selected image type to the data 
table for future processing. In addition, processor 106 auto- 
matically fills in default parameter values and tools in table 
200 for the newly selected image type. If desired the default 
values for the newly selected image type can be modified as 
explained below. To remove an image type from table 370, 
the image type is selected and in table 370 delete icon 374 
is selected. 

Referring to FIGS. 2 and 9, to modify a default position 
image corresponding to Dr. Welby's C-spine exam, the 
technologist selects one of the image types in table 370 and 
thereafter selects modify images icon 362, Referring also to 
FIG. 10, when the modify images icon 362 is selected, 
processor 106 divides space 254 into a plurality of spaces 
and provides a separate position image in each of the 
plurality of spaces. In FIG. 10, space 254 is divided into nine 
separate spaces and therefore nine separate position images 
are provided. Since additional position images are possible, 
additional screens may be accessed via a scrolling tool (not 
illustrated) in space 254. Preferably, a position image is 
provided for each image in position image table 202 in FIG. 
2. A separate selection icon (i.e. P-001, P-002, etc.) Is 
provided for each displayed image. The technologist can 
select any of the position images illustrated in space 254 via 
icon 228 and a properly selected icon P-001, P-002, etc. 
After selecting an image, the technologist selects the accept 
icon 366 at which point processor 106 updates data table 200 
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to indicate the selected position image in column 210. 
Thereafter, processor 106 regenerates the screen illustrated 
in FIG. 9. 

Referring again to FIGS. 2 and 9, to modify the prefer- 
ences and tools in column 214 of iM& 200, a technologist 
selects modify preferences icon 364. AUhough not 
illustrated, when icon 364 is selected, processor 106 indi- 
cates all parameter values in space 254 and provides suitable 
iconic tools for modifying the parameter values. Once the 
parameter values have been set, the technologist again 
accepts the values by selecting an icon similar to accept icon 
366 which returns the technologist back to the screen 
illustrated in FIG. 9. 

Referring still to FIGS. 2 and 9, to accept all of the 
changes made to table 200, the technologist selects accept 
icon 366 once again at which point processor 106 provides 
the screen illustrated in FIG. 8, Other modifications to 
protocols can be made in a similar fashion. 

It should be appreciated that the above-described interface 
can be used to simplify technologist training thereby 
enabling less expensive technologist cross-training between 
imaging system modalities and enabling interfacility tech- 
nologist substitution with relatively little difficulty. The 
guidance tools which consist primarily of guidance tables 
similar to those illustrated in FIG. 2 and an easily understood 
workflow which is presented in function and workflow icons 
through the inventive universal interface enable easy cross- 
training and interfacihty substitutions. 

To apprise the public of the scope of this invention, we 
make the following claims. 

What is claimed is: 

1. A medical image data acquisition system interface 
usable with at least first and second different medical imag- 
ing data acquisition systems that include first and second 
data acquisition hardware configurations for acquiring data 
using different first and second medical image data acqui- 
sition modalities, respectively, wherein the first and second 
modalities may be selected from radiography, fluoroscopy, 
angiography, magnetic resonance imaging, ultrasound, 
nuclear medicine, positron emission tomography and com- 
puter tomography, each modality including functions which 
are common to each of the first and second modalities, each 
separate instance of the interface used with only a specific 
one of the first or second modality systems and linked to a 
diagnostic system for acquiring data according to the spe- 
cific one of the first and second acquisition modalities, the 
interface comprising: 

a display; 

a programmed data processor for providing a uniform 
interface image on the display despite the specific data 
acquisition modality associated with the diagnostic 
system linked to the interface, the uniform interface 
image comprising: 

a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality, wherein the workspace includes a work- 
flow navigation space in which, when a function icon 
is selected, the processor displays a workflow icons 
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set including a separate workflow icon correspond- 
ing to each subprocess of the process associated with 
the selected function icon and for the specific modal- 
ity; and 

a pointing device for moving a pointer icon about the 5 
display and for selecting displayed icons; 

wherein, when an icon is selected, the processor cor- 
relates the selected icon with a corresponding data 
acquisition command and executes the command; 

. 10 

wherein each subprocess includes subprocess specific 
parameters, the workspace includes both an imaging 
window and a parameter setting space adjacent the 
imaging window and, when a workflow icon is 
selected, the processor displays a parameter value set 
and setting icons in the setting space which corre- 
spond to the subprocess specific parameters associ- 
ated with the selected workflow icon, each parameter 
value indicating the current parameter value, the 
setting icons useable to modify the current parameter 
values. 20 

2. The interface of claim 1 wherein functions correspond- 
ing to each specific one of the medical image data acquisi- 
tion modalities are typically performed in an exemplary 
workflow pattern including a series of sequential steps and 
the function icons are arranged in an order of the exemplary 25 
workflow pattern. 

3. The interface of claim 2 wherein the function icons are 
arranged in a single column. 

4. The interface of claim 2 wherein the function icons 
include a patient information icon and an acquire icon. 30 

5. The interface of claim 4 wherein the function icons also 
include a view/film icon, an analyze icon and a close icon. 

6. The interface of claim 1 wherein the subprocesses of 
each procedure typically are performed in an exemplary 
workflow pattern including a series of consecutive steps and 35 
the workflow icons are arranged in an order of the exemplary 
workflow pattern, 

7. The interface of claim 6, wherein the workflow icons 
are arranged in a single column. 

8. The interface of claim 1 wherein the workflow icons 40 
include a setup icon which corresponds to the process of 
positioning a patient for imaging, when the setup icon is 
selected, the processor providing a position window in the 
workspace and providing an image of a properly positioned 
patient in the position window. 45 

9. Interface of claim 1 wherein subprocess parameters 
corresponding to each subprocess include a commonly 
modified parameter subset and a seldomly modified param- 
eter subset and wherein, when a workflow icon is selected 
the processor displays parameter values and setting icons 50 
which correspond to the commonly modified parameter 
subset and a toolbox icon, when the toolbox icon is selected, 
the processor opening a toolbox window and displaying the 
parameter values and setting icons which correspond to the 
seldomly modified parameter set. 55 

10. The interface of claim 1 wherein a plurality of 
physicians may prescribe imaging using the specific modal- 
ity and each physician may have different initial current 
parameter value settings for each subprocess, the processor 
storing a table which correlates each physician with physi- 60 
cian specific initial current parameter value settings, a phy- 
sician identifier provided to the processor and, when a 
specific workflow icon is selected, the processor accesses the 
table, correlates the physician identifier with the selected 
subprocess initial current parameter value settings and pro- 65 
vides parameter icons indicating the initial current parameter 
value settings. 
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11. The interface of claim 1 wherein, in addition to the 
navigation space and the workspace, the processor also 
provides a domain space and a schedule icon in the domain 
space and wherein, when the schedule con is selected, the 
processor clears the workspace and navigation space and 
therein provides a schedule metaphor image which includes 
imaging system scheduling information. 

12. The interface of claim 1 wherein, in addition to the 
navigation space and the workspace, the processor also 
provides a domain space and a logbook icon in the domain 
space and wherein, when the logbook icon is selected, the 
processor clears the workspace and navigation space and 
therein provides a logbook metaphor image which includes 
imaging system logbook information. 

13. The interface of claim 1 wherein the system is also an 
image manipulation system. 

14. A medical image data acquisition system interface 
usable with at least first and second different medical imag- 
ing data acquisition systems that include first and second 
data acquisition hardware configurations for acquiring data 
using different first and second medical image data acqui- 
sition modalities, respectively, wherein the first and second 
modalities may be selected from radiography, fluoroscopy, 
angiography, magnetic resonance imaging, ultrasound, 
nuclear medicine, positron emission tomography and com- 
puter tomography, each modality including functions which 
are common to each of the first and second modalities, each 
separate instance of the interface used with only a specific 
one of the first or second modaflty systems and linked to a 
diagnostic system for acquiring data according to the spe- 
cific one of the first and second acquisition modalities, the 
interface comprising: 

a display; 

a programmed data processor for providing a uniform 
interface image on the display despite the specific data 
acquisition modality associated with the diagnostic 
system linked to the interface, the uniform interface 
image comprising: 

a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 

common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality, wherein the workspace includes a work- 
flow navigation space in which, when a function icon 
is selected, the processor displays a workflow icons 
set including a separate workflow icon correspond- 
ing to each subprocess of the process associated with 
the selected function icon and for the specific modal- 
ity; and 

a pointing device for moving a pointer icon about the 
display and for selecting displayed icons; 

wherein, when an icon is selected, the processor cor- 
relates the selected icon with a corresponding data 
acquisition command and executes the command; 
and 

wherein the processor stores a workflow table which 
correlates subprocess sets with each modality func- 
tion and wherein the processor provides a protocols 
icon which, when selected, causes the processor to 
display icons in the workspace for modifying the 
subprocess set to provide a modified subprocess set 
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which is then stored by the processor as the subpro- 
cess set, the next time the corresponding function 
icon is selected, the processor providing a workflow 
icon for each of the subprocesses in the subprocess 
set. 

15. A medical image data acquisition system interface 
usable with at least first and second different medical imag- 
ing data acquisition systems that include first and second 
data acquisition hardware configurations for acquiring data 
using different first and second medical image data acqui- 
sition modalities, respectively, wherein the first and second 
modalities may be selected from radiography, fluoroscopy, 
angiography, magnetic resonance imaging, ultrasound, 
nuclear medicine, positron emission tomography and com- 
puter tomography, each modality including functions which 
are common to each of the first and second modalities, each 
separate instance of the interface used with only a specific 
one of the first or second modality systems and linked to a 
diagnostic system for acquiring data according to the spe- 
cific one of the first and second acquisition modalities, the 
interface comprising: 

a display; 

a programmed data processor for providing a uniform 
interface image on the display despite the specific data 
acquisition modality associated with the diagnostic 
system linked to the interface, the uniform interface 
image comprising: 

a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent tiie function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality, wherein the wortepace includes a work- 
flow navigation space in which, when a function icon 
is selected, the processor displays a workflow icons 
set including a separate workflow icon correspond- 
ing to each subprocess of the process associated with 
the selected function icon and for the specific modal- 
ity; and 

a pointing device for moving a pointer icon about the 
display and for selecting displayed icons; 

wherein, when an icon is selected, the processor cor- 
relates the selected icon with a corresponding data 
acquisition command and executes the command; 
and 

wherein a plurality of physicians may prescribe imag- 
ing using the specific modality and each physician 
may have a different subprocess procedure for 
modality functions, the processor storing a table 
which correlates each physician with a physician 
specific subprocess set for each function, a physician 
identifier provided to the processor and, when a 
specific function icon Is selected, the processor 
accesses the table, correlates the physician identifier 
with selected function subprocess set and provides a 
workflow icon for each subprocess in the subprocess 
set. 

16. A method for use with a specific one of at least first 
and second imaging systems, the first and second systems 
including first and second data acquisition hardware con- 
figurations for acquiring data using different first and second 
medical image data acquisition modalities, respectively, 



wherein the first and second modalities may be selected 
from radiography, fluoroscopy, angiography, magnetic reso- 
nance imaging, ultrasound, nuclear medicine, positron emis- 
sion tomography and computer tomography, each system 
5 also including a processor linked to the acquisition 
hardware, a display and a pointing device for moving a 
pointer icon about the display and for selecting displayed 
icons, the method for guiding a technologist through the 
processes required to use a modality specific acquisition 
10 system, the method comprising the steps of: 

providing an interface image on the display including: 
a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
15 sition modalities wherein, each procedure which is 

common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
20 displaying, analyzing and manipulating images of a 

type consistent with the specific data acquisition 
modality; 

providing a pointer icon on the display; 
when the pointer icons is used to select one of the 
25 function icons, correlating the selected icon with a 

corresponding data acquisition command and 
executes the command, 
wherein the step of providing an interface image 
includes the step of, as part of the workspace, 
30 providing a workflow navigation space and, when a 

function icon is selected, the method further includes 
the step of displaying a workflow icon set including 
a separate workflow icon corresponding to each 
subprocess of the process associated with the 
35 selected function icon and for the specific modality 

in the workflow navigation space; and 
wherein each subprocess includes subprocess specific 
parameters, the step of providing an interface image 
includes the step of, as part of the workspace, 
40 providing both an imaging window and a parameter 

setting space adjacent the imaging window and, 
when a workflow icon is selected, the method further 
includes the steps of, displaying a parameter value 
set and setting icons in the setting space which 
45 correspond to the subprocess specific parameters 

associated with the selected workflow icon, each 
parameter value indicating the current parameter 
value, the setting icons useable to modify the current 
parameter values. 
50 17. The method of claim 16 wherein functions corre- 
sponding to each specific one of the medical image data 
acquisition modalities are typically performed in an exem- 
plary workflow pattern including a series of sequential steps 
and the function icons are arranged in an order of the 
55 exemplary workflow pattern. 

18. The method of claim 17 wherein the function icons are 
arranged in a single column. 

19. The method of claim 16 wherein the subprocesses of 
each procedure typically are performed in an exemplary 

60 workflow pattern including a series of consecutive steps and 
the workflow icons are arranged in the workflow navigation 
space in an order of the exemplary workflow pattern. 

20. The method of claim 16 wherein a plurality of 
physicians may prescribe imaging using the specific modal- 

65 ity and each physician may have different initial current 
parameter value settings for each subprocess, the processor 
storing a table which correlates each physician with physi- 
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cian specific initial current paranaeter value settings, the 
method further including the steps of receiving a physician 
identifier, when a specific workflow icon is selected, access- 
ing the table, correlating the physician identifier with the 
selected subprocess initial current parameter value settings 5 
and providing parameter icons indicating the initial current 
parameter value settings. 

21. The method of claim 16 wherein the system is also an 
image manipulation system, 

22. A method for use with a specific one of at least first 10 
and second imaging systems, the first and second systems 
including first and second data acquisition hardware con- 
figurations for acquiring data using different first and second 
medical image data acquisition modalities, respectively, 
wherein the first and second modalities may be selected 15 
from radiography, fluoroscopy, angiography, magnetic reso- 
nance imaging, ultrasound, nuclear medicine, positron emis- 
sion tomography and computer tomography, each system 
also including a processor linked to the acquisition 
hardware, a display and a pointing device for moving a 20 
pointer icon about the display and for selecting displayed 
icons, the method for guiding a technologist through the 
processes required to use a modality specific acquisition 
system, the method comprising the steps of: 

providing an interface image on the display including: 25 
a fiinction navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 30 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 35 
modality; 

providing a pointer icon on the display; 

when the pointer icons is used to select one of the 
function icons, correlating the selected icon with a 
corresponding data acquisition command and 40 
executes the command, 

wherein the step of providing an interface image 
includes the step of, as part of the workspace, 
providing a workflow navigation space and, when a 
function icon is selected, the method further includes 45 
the step of displaying a workflow icon set including 
a separate workflow icon corresponding to each 
subprocess of the process associated with the 
selected function icon and for the specific modality 
in the workflow navigation space; and 50 

wherein the workflow icons include a setup icon which 
corresponds to the process of positioning a patient 
for imaging, when the setup icon is selected, the 
method further including the step of providing a 
position window in the workspace and providing an 55 
image of a properly positioned patient in the position 
window. 

23. A method for use with a specific one of at least first 
and second imaging systems, the first and second systems 
including first and second data acquisition hardware con- 60 
figurations for acquiring data using different first and second 
medical image data acquisition modalities, respectively, 
wherein the first and second modalities may be selected 
from radiography, fluoroscopy, angiography, magnetic reso- 
nance imaging, ultrasound, nuclear medicine, positron emis- 65 
sion tomography and computer tomography, each system 
also including a processor linked to the acquisition 



hardware, a display and a pointing device for moving a 
pointer icon about the display and for selecting displayed 
icons, the method for guiding a technologist through the 
processes required to use a modality specific acquisition 
system, the method comprising the steps of: 

providing an interface image on the display including: 
a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality; 

providing a pointer icon on the display; 
when the pointer icons is used to select one of the 
function icons, correlating the selected icon with a 
corresponding data acquisition command and 
executes the command, 
wherein the step of providing an interface image 
includes the step of, as part of the workspace, 
providing a workflow navigation space and, when a 
function icon is selected, the method further includes 
the step of displaying a workflow icon set including 
a separate workflow icon corresponding to each 
subprocess of the process associated with the 
selected function icon and for the specific modality 
in the workflow navigation space; and 
wherein subprocess parameters corresponding to each 
subprocess include a commonly modified parameter 
subset and a seldomly modified parameter subset and 
wherein, when a workflow icon is selected the step 
of displaying parameter values includes the step of 
displaying parameter values and setting icons which 
correspond to the commonly modified parameter 
subset and a toolbox icon and, when the toolbox icon 
is selected, the method further includes the step of 
opening a toolbox window and displaying the 
parameter values and setting icons which correspond 
to the seldomly modified parameter set. 
24. A method for use with a specific one of at least first 
and second imaging systems, the first and second systems 
including first and second data acquisition hardware con- 
figurations for acquiring data using different first and second 
medical image data acquisition modalities, respectively, 
wherein the first and second modalities may be selected 
firom radiography, fluoroscopy, angiography, magnetic reso- 
nance imaging, ultrasound, nuclear medicine, positron emis- 
sion tomography and computer tomography, each system 
also including a processor linked to the acquisition 
hardware, a display and a pointing device for moving a 
pointer icon about the display and for selecting displayed 
icons, the method for guiding a technologist through the 
processes required to use a modality specific acquisition 
system, the method comprising the steps of: 
providing an interface image on the display including: 
a function navigation space including ftinction icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality; 
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providing a pointer icon on the display; 
when the pointer icons is used to select one of the 
function icons, correlating the selected icon with a 
corresponding data acquisition command and 
executes the command, 
wherein the step of providing an interface image 
includes the step of, as part of the workspace, 
providing a workflow navigation space and, when a 
function icon is selected, the method further includes 
the step of displaying a workflow icon set including 
a separate workflow icon corresponding to each 
subprocess of the process associated with the 
selected function icon and for the specific modality 
in the workflow navigation space; and 
wherein the processor stores a workflow table which 
correlates subprocess sets with each modality func- 
tion and wherein the method further includes the 
steps of displaying a protocols icon, when the pro- 
tocols icon is selected, displaying icons in the work- 
space for modifying the subprocess set to provide a 
modified subprocess set, storing the modified sub- 
process set as the subprocess set and, the next time 
the corresponding function icon is selected, provid- 
ing a workflow icon for each of the subprocesses in 
the subprocess set. 
25, A method for use with a specific one of at least first 
and second imaging systems, the first and second systems 
including first and second data acquisition hardware con- 
figurations for acquiring data using different first and second 
medical image data acquisition modalities, respectively, 
wherein the first and second modalities may be selected 
from radiography, fluoroscopy, angiography, magnetic reso- 
nance imaging, tiltrasotmd, nuclear medicine, positron emis- 
sion tomography and computer tomography, each system 
also including a processor linked to the acquisition 
hardware, a display and a pointing device for moving a 
pointer icon about the display and for selecting displayed 
icons, the method for guiding a technologist through the 
processes required to use a modality specific acquisition 
system, the method comprising the steps of: 
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providing an interface image on the display including: 
a function navigation space including function icons 
corresponding to data acquisition procedures which 
are common to both the first and second data acqui- 
sition modalities wherein, each procedure which is 
common to the first and second data acquisition 
modalities includes procedure-specific subprocesses; 
and 

a workspace adjacent the function navigation space for 
displaying, analyzing and manipulating images of a 
type consistent with the specific data acquisition 
modality; 

providing a pointer icon on the display; 

when the pointer icons is used to select one of the 
function icons, correlating the selected icon with a 
corresponding data acquisition command and 

executes the command, 

wherein the step of providing an interface image 
includes the step of, as part of the workspace, 
providing a workflow navigation space and, when a 
function icon is selected, the method further includes 
the step of displaying a workflow icon set including 
a separate workflow icon corresponding to each 
subprocess of the process associated with the 
selected function icon and for the specific modality 
in the workflow navigation space; and 

wherein a plurality of physicians may prescribe imag- 
ing using the specific modality and each physician 
may have a different subprocess procedure for 
modality functions, the processor storing a table 
which correlates each physician with a physician 
specific subprocess set for each function, the method 
further including the steps of receiving a physician 
identifier and, when a specific function icon is 
selected, accessing the table, correlating the physi- 
cian identifier with the selected function subprocess 
set and providing a workflow icon for each subpro- 
cess in the subprocess set. 



